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2015  WORKPLACE  AND  GENDER  RELATIONS  SURVEY  OF 
RESERVE  COMPONENT  MEMBERS: 
STATISTICAL  METHODOLOGY  REPORT 

Introduction 

This  report  describes  the  statistical  methodologies  for  the  2015  Workplace  and  Gender 
Relations  Survey  of  Reserve  Component  Members  ( 2015  WGRR).  The  first  section  describes  the 
sample  design  and  selection  of  the  sample.  The  second  section  describes  weighting  and  variance 
estimation,  as  well  as  a  comparison  to  the  2014  RAND  Military  Workplace  Study.  The  third 
section  describes  the  statistical  tests  used  for  the  2015  WGRR.  The  fourth  section  describes  the 
calculation  of  location,  completion,  and  response  rates  for  the  full  sample  and  population 
subgroups.  The  final  section  contains  the  nonresponse  bias  (NRB)  analysis.  Estimates  for  all 
survey  questions  are  found  in  the  2015  Workplace  and  Gender  Relations  Survey  of  Reserve 
Component  Members:  Tabulation  Volume  (DMDC,  2016a). 

Sample  Design  and  Selection 


Target  Population 

The  2015  WGRR  was  designed  to  represent  individuals  meeting  the  following  criteria: 

•  Members  of  the  Selected  Reserve  who  are  in  Reserve  Unit,  Active  Guard/Reserve 
(AGR/FTS/AR;  Title  10  and  Title  32),  and  Individual  Mobilization  Augmentee 
(IMA)  programs  from: 

o  Army  National  Guard  (ARNG), 

o  US  Army  Reserve  (USAR), 

o  US  Navy  Reserve  (USNR), 

o  US  Marine  Corps  Reserve  (USMCR), 

o  Air  National  Guard  (ANG),  or 

o  US  Air  Force  Reserve  (USAFR); 

•  Up  to  and  including  paygrade  06  as  of  March  2015;  Reserve  component  members 
who  entered  the  Service  after  March  2015  are  excluded  from  the  population. 

•  The  sampling  frame  was  developed  five  months  prior  to  fielding  the  survey  so  the 
sampling  population  included  those  that  had  been  in  the  Selected  Reserve  for  at  least 
five  months. 
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Data  were  collected  on  the  web  between  August  7,  2015  and  October  19,  2015.  If 
sample  members  had  not  responded  within  the  first  month  of  the  fielding  period,  they 
were  sent  the  paper-and-pen  survey. 

Sampling  Frame 

The  sampling  frame  consisted  of  817,007  Reserve  component  members  using  the  March 
2015  Reserve  Components  Common  Personnel  Data  System  (RCCPDS)  Master  File.  Auxiliary 
frame  data  was  obtained  from  the  following  files: 

•  March  2015  Reserve  Family  Database  File  (contains  the  member’s  family 
information,  (e.g.  marital  status  and  children)) 

•  March  2015  Contingency  Tracking  System  (CTS)  File  (contains  deployment 
information) 

•  April  2015  Defense  Enrollment  Eligibility  Reporting  System  (DEERS)  Medical 
Point-In-Time  Extract  (PITE)  (contains  personnel  information) 

•  Time  on  Active  Duty  (TOAD)  File,  pulled  August  2015  (contains  activation 
information) 

In  addition,  after  selecting  the  sample,  DMDC  performed  additional  checks  to  verily  the 
member  was  still  eligible  before  the  survey  fielded.  Any  ineligible  member  in  the  sample  was 
excluded  from  any  further  mailings  and  notifications;  this  saved  additional  costs  associated  with 
the  survey  process.  Using  the  May  2014  RCCPDS,  DMDC  detennined  10,630  sample  members 
(2.2  percent  unweighted)  were  record  ineligible  and  excluded  them  from  mailings  and 
notifications  (see  Table  3). 

Sample  Design 

The  sample  for  the  2015  WGRR  survey  used  a  single-stage  stratified  design.  Four 
population  characteristics  defined  the  stratification  dimensions  for  the  2015  WGRR  sample: 

•  Reserve  component  (Army  National  Guard,  Army  Reserve,  Navy  Reserve,  Marine 
Corps  Reserve,  Air  National  Guard,  Air  Force  Reserve) 

•  Gender  (Male,  Female), 

•  Paygrade  grouping  (E1-E4,  E5-E9,  W1-W5,  01-03,  04-06),  and 

•  Reserve  program  (Troop  Program  Unit  [TPU],  Active  Guard/Reserve  [AGR], 
Military  Technician  [MilTech],  and  Individual  Mobilization  Augmentee  [IMA]). 

Table  1  shows  these  four  variables  and  associated  variable  levels. 
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Table  1. 

Stratifying  Variables 


Variable 

Variable  Name 

Categories 

Reserve  Component 

RORGCD 

1.  Army  National  Guard 

2.  US  Army  Reserve 

3.  US  Navy  Reserve 

4.  US  Marine  Corps  Reserve 

5.  Air  National  Guard 

6.  US  Air  Force  Reserve 

Gender 

RSEX2 

1.  Male 

2.  Female 

Paygrade  Grouping 

RPAYGRP9 

1.E1-E4 

2.  E5-E9 

3.  W1-W5 

4.  01-03 

5.  04-06 

Reserve  Program 

RPROG1 

l.TPU 

2.  AGR 

3.  MilTech 

4.  IMA 

DMDC  partitioned  the  population  frame  of  817,007  members  into  128  strata  that  were 
initially  determined  by  a  full  cross-classification  of  the  four  stratification  variables.  Levels  were 
collapsed  when  there  were  less  than  200  members  in  the  stratum,  usually  for  Reserve  program 
and  rarely  paygrade  grouping.  Dimensions  within  Reserve  component  and  gender  were  always 
preserved. 

DMDC  selected  individuals  with  equal  probability  and  without  replacement  within  each 
stratum.  However,  because  allocation  was  not  proportional  to  the  size  of  the  strata,  selection 
probabilities  varied  among  strata  and  individuals  were  not  selected  with  equal  probability 
overall.  To  achieve  adequate  sample  sizes  for  all  domains  (reporting  categories)  DMDC  used  a 
non-proportional  allocation.  Appendix  A  shows  the  estimation  domains  along  with  their  sample 
sizes,  expected  number  of  respondents,  and  estimated  precisions. 

Sample  Allocation 

DMDC  based  the  total  sample  size  on  a  census  of  females  and  50  percent  sample  of 
males.  The  goal  was  to  achieve  reliable  precision  on  estimates  for  outcomes  associated  with 
reporting  a  sexual  assault  (i.e.,  retaliation)  and  other  measures  that  were  only  asked  of  a  very 
small  subset  of  members,  especially  for  males.  Given  estimated  variable  survey  costs  and 
anticipated  eligibility  and  response  rates,  DMDC  used  an  optimization  algorithm  to  determine 
the  minimum-cost  allocation  that  simultaneously  satisfied  the  domain  precision  requirements. 
Response  rates  from  previous  surveys  were  used  to  estimate  eligibility  and  response  rates  for  all 
strata.  The  2013  Status  of  Forces  Survey  of  Reserve  Component  Members,  the  2014  Status  of 
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Forces  Survey  of  Reserve  Component  Members,  and  the  2012  Workplace  and  Gender  Relations 
Survey  of  Reserve  Component  Members  were  used  to  estimate  these  rates. 

DMDC  detennined  the  sample  allocation  given  the  census  of  females  and  50  percent  of 
males  by  means  of  the  DMDC  Sample  Planning  Tool  (SPT),  Version  2.1  (Dever  &  Mason, 

2003).  This  application  is  based  on  the  method  originally  developed  by  J.  R.  Chromy  (1987)  and 
described  in  Mason,  Wheeless,  George,  Dever,  Riemer,  and  Elig  (1995).  The  SPT  defines 
domain  variance  equations  in  terms  of  unknown  stratum  sample  sizes  and  user-specified 
precision  constraints.  A  cost  function  is  defined  in  terms  of  the  unknown  stratum  sample  sizes 
and  the  per-unit  cost  of  data  collection,  editing,  and  processing.  The  variance  equations  are 
solved  simultaneously,  subject  to  the  constraints  imposed,  for  the  sample  size  that  minimizes  the 
cost  function.  Estimated  eligibility  rates  are  used  and  they  modify  the  estimated  prevalence  rates 
used  in  the  variance  equations,  thus  affecting  the  allocation;  response  rates  inflate  the  allocation, 
thus  affecting  the  final  sample  size.  Prevalence  rates  refer  to  a  percentage  that  is  used  in 
determining  the  estimated  variance  used  for  the  calculation  of  the  sample  size.  For  example, 
DMDC  used  50  percent  since  it  is  most  conservative  and  yields  the  largest  estimated  sample  size. 

There  were  93  reporting  domains  defined  for  the  2015  WGRR  and  the  initial  goal  was  to 
achieve  below  5  percent  precision  on  estimates.  The  precision  requirement  for  each  domain  is 
typically  based  on  an  estimated  prevalence  rate  of  0.5  with  a  95  percent  confidence  interval  half¬ 
width  no  greater  than  0.05.  However,  given  the  rarity  of  events  covered  by  many  of  the  2015 
WGRR  questions,  DMDC  ensured  that  a  much  tighter  precision  would  be  met  for  questions  seen 
by  all  respondents,  while  making  it  likely  that  confidence  interval  half-widths  of  0.05  could  be 
met  for  questions  that  are  relevant  to  only  a  small  portion  of  respondents.  Therefore,  DMDC 
tightened  the  precision  constraints  until  the  sample  included  a  census  of  all  females  and  at  least 
50  percent  of  all  males. 

The  2015  WGRR  total  sample  size  was  485,774;  Table  2  provides  the  sample  sizes  by 
stratification  variables. 
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Table  2. 

Sample  Size  by  Stratification  Variables 


Stratification 

Variable 

Total 

Army  National 
Guard 

US  Army 
Reserve 

US  Navy 
Reserve 

US  Marine 
Corps  Reserve 

Air  National 
Guard 

US  Air  Force 
Reserve 

Sample 

485,774 

186,481 

121,036 

36,245 

36,364 

61,695 

43,953 

Gender 

Male 

331,332 

130,498 

75,898 

23,406 

34,750 

41,090 

25,690 

Female 

154,442 

55,983 

45,138 

12,839 

1,614 

20,605 

18,263 

Paygrade  Grouping 

E1-E4 

238,102 

104,338 

62,526 

9,995 

25,784 

20,656 

14,803 

E5-E9 

179,140 

62,876 

39,436 

18,417 

6,315 

31,745 

20,351 

W1-W5 

5,773 

3,823 

1,651 

40 

259 

0 

0 

01-03 

33,684 

11,143 

10,659 

3,235 

1,729 

4,002 

2,916 

04-06 

29,075 

4,301 

6,764 

4,558 

2,277 

5,292 

5,883 

Reserve  Program 

TPU 

414,431 

167,140 

109,335 

30,467 

32,111 

41,924 

33,454 

AGR 

33,432 

10,010 

6,890 

5,737 

1,621 

8,132 

1,042 

MilTech 

29,273 

9,331 

3,213 

0 

0 

11,639 

5,090 

IMA 

8,638 

0 

1,598 

41 

2,632 

0 

4,367 

Survey  Administration 

Information  about  administration  of  the  survey  and  detailed  documentation  of  the  survey 
dataset  are  found  in  the  2015  Workplace  and  Gender  Relations  Survey  of  Reserve  Component 
Members:  Administration,  Datasets,  and  Codebook  (DMDC,  2016b). 

Weighting 

Analytical  weights  for  the  2015  WGRR  were  created  to  account  for  unequal  probabilities 
of  selection  and  varying  response  rates  among  population  subgroups.  Sampling  weights  were 
computed  as  the  inverse  of  the  selection  probabilities.  The  sampling  weights  were  then  adjusted 
for  nonresponse  using  models  that  considered  over  50  possible  correlates  of  nonresponse.  The 
adjusted  weights  were  post-stratified  to  match  population  totals  and  to  reduce  bias  unaccounted 
for  by  the  previous  weighting  steps. 

Case  Dispositions 

As  the  first  step  in  the  weighting  process,  case  dispositions  were  assigned  based  on 
eligibility  for  the  survey  and  on  completion  of  the  questionnaire.  Execution  of  the  weighting 
process  and  computation  of  response  rates  both  depended  on  this  classification. 

Final  case  dispositions  for  weighting  were  determined  using  information  from  personnel 
records,  field  operations  (as  recorded  in  the  Survey  Control  System  [SCS]),  and  returned 
questionnaires.  No  single  source  of  information  is  entirely  complete  and  correct  for  determining 
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the  case  disposition;  inconsistencies  among  sources  were  resolved  according  to  the  order  of 
precedence  shown  in  Table  3.  This  order  of  execution  is  critical  to  resolving  case  dispositions. 
For  example,  suppose  an  individual  in  the  sample  refused  the  survey,  with  the  reason  that  it  was 
too  long;  in  the  absence  of  any  other  infonnation,  the  disposition  would  be  “eligible 
nonrespondent.”  Another  example  would  be  if  we  were  provided  a  proxy  report  that  the  sample 
member  had  been  hospitalized  and  was  unable  to  complete  the  survey,  in  this  instance  the 
disposition  would  be  “ineligible.” 

Case  disposition  counts  for  the  2015  WGRR  are  shown  in  Table  3.  Table  4  presents  the 
number  of  complete  eligible  respondents  (SAMP_DC=4)  by  stratification  variables:  gender, 
paygrade  groups,  reserve  programs,  and  reserve  organizations. 
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Table  3. 

Case  Dispositions  for  Weighting 


Case  Disposition 
(SAMP_DC) 

Information 

Source 

Conditions 

Eligibility  Known 

Sample 

Size 

1.  Record  ineligible 

Personnel  record 

DMDC  determined  whether  sampled 
members  had  a  record  in  the  DEERS  point- 
in-time  extract  (PITE)  prior  to  fielding  the 
survey.  No  record  in  DEERS  indicated  the 
member  either  separated  from  the  military, 
passed  away,  etc. 

NA 

10,630 

2.  Ineligible  by  self-  or 
proxy-report 

Survey  Control 
System  (SCS) 

The  sampled  member  or  a  proxy  reported 
that  member  was  ineligible  due  to  such 
reasons  as  "Retired,"  “111,”  “Incarcerated,” 
“No  longer  employed  by  DoD,”  or 
“Deceased.” 

Yes 

210 

3.  Ineligible  by  survey 
self-report 

Survey  eligibility 
question 

The  sampled  member  was  determined  to  be 
ineligible  based  on  their  response  to  Q1  of 
the  survey  questionnaire  asking  if  retired  or 
separated. 

Yes 

1,331 

4.  Eligible,  complete 
response 

Item  response  rate 

Respondents  needed  to  answer  one  of  the 
six  critical  questions  related  to  sexual 
assault. 

Yes 

87,127 

5.  Eligible,  incomplete 
response 

Item  response  rate 

Survey  is  not  blank  but  none  of  the  critical 
sexual  assault  questions  were  answered. 

Yes 

1,985 

8.  Refusal 

SCS 

Survey  is  returned  blank  due  to  such 
reasons  as  “Refused-too  long,”  “Refused- 
inappropriate/intrusive,”  “Refused-other,” 
“Unreachable  at  this  address,”  “Refused  by 
current  resident,”  “Refused  additional  e- 
mails,”  or  “Concerned  about 
security/confidentiality.” 

No 

1,176 

9.  Blank  return 

SCS 

Blank  questionnaire  returned  with  no 
reason  given. 

No 

611 

10.  PND 

SCS 

Postal  non-deliverable  or  original  address  is 
non-locatable. 

No 

46,592 

11.  Nonrespondent 

Remainder 

Remaining  sampled  members  did  not 
respond  to  survey. 

No 

336,112 

Total 

485,774 
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Table  4. 

Complete  Eligible  Respondents  by  Stratification  Variables 


Stratification 

Variable 

Total 

Army 

National 

Guard 

US  Army 
Reserve 

US  Navy 
Reserve 

US  Marine 
Corps 
Reserve 

Air  National 
Guard 

US  Air 
Force 
Reserve 

Sample 

87,127 

25,172 

18,674 

8,053 

4,002 

19,195 

12,031 

Gender 

Male 

52,421 

15,329 

10,288 

5,028 

3,673 

11,730 

6,373 

Female 

34,706 

9,843 

8,386 

3,025 

329 

7,465 

5,658 

Paygrade  Grouping 

E1-E4 

18,575 

6,096 

3,834 

1,034 

1,883 

3,512 

2,216 

E5-E9 

45,178 

12,798 

8,807 

4,095 

1,022 

11,972 

6,484 

W1-W5 

2,203 

1,425 

667 

21 

90 

0 

0 

01-03 

9,125 

2,830 

2,642 

1,014 

334 

1,422 

883 

04-06 

12,046 

2,023 

2,724 

1,889 

673 

2,289 

2,448 

Reserve  Program 

TPU 

57,902 

16,526 

14,166 

6,514 

2,822 

9,916 

7,958 

AGR 

13,773 

4,711 

2,711 

1,519 

511 

3,876 

445 

MilTech 

12,737 

3,935 

1,246 

0 

0 

5,403 

2,153 

IMA 

2,715 

0 

551 

20 

669 

0 

1,475 

Nonresponse  Adjustments  and  Final  Weights 

After  case  dispositions  were  resolved,  the  sampling  weights  were  adjusted  for 
nonresponse.  First,  the  sampling  weights  for  cases  of  known  eligibility  (SAMP_DC  =  2,  3,  4,  or 
5)  were  adjusted  to  account  for  cases  of  unknown  eligibility  (SAMPDC  =  8,  9,  10,  or  1 1). 

Next,  the  eligibility-adjusted  weights  for  eligible  respondents  with  completed  questionnaires 
(SAMP  DC  =  4)  were  adjusted  to  account  for  eligible  sample  members  who  returned  an 
incomplete  questionnaire  (SAMP  DC  =  5).  All  weights  for  the  record  ineligibles 
(SAMP_DC=1)  are  set  to  0  and  this  weight  is  transferred  to  the  other  cases. 

The  weighting  adjustment  factors  for  eligibility  and  completion  were  computed  as  the 
inverse  of  model-predicted  probabilities.  The  2015  models  used  to  predict  these  probabilities 
changed  substantially  from  the  models  for  the  2012  WGRR.  The  2015  models  paralleled  those 
developed  by  RAND  for  the  2014  RAND  Military  Workplace  Study  (2014  RMWS)  (Morral, 

Gore,  &  Schell,  2014,  2015),  which  surveyed  both  the  active  duty  and  Reserve  members.  The 
sample  size  for  the  2015  WGRR ,  485,774,  was  considerably  larger  than  either  the  sample  size  for 
the  2012  WGRR ,  75,436,  or  the  Reserve  sample  size  in  the  2014  RMWS,  67,559. 

For  the  2012  WGRR ,  a  logistic  regression  model  was  used  to  predict  the  probability  of 
known  eligibility  for  the  survey  (that  is,  the  probability  of  determining  eligibility).  A  second 
logistic  regression  model  was  used  to  predict  the  probability  of  response  among  eligible  sample 
members  (complete  response  vs.  nonresponse).  CHAID  (Chi-squared  Automatic  Interaction 
Detector)  was  used  to  determine  the  best  predictors  for  each  logistic  regression  model.  The 
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models  were  weighted.  Predictors  included  the  following  population  characteristics:  paygrade 
group,  gender,  reserve  program,  reserve  component,  education,  family  status,  combat/non¬ 
combat  flag,  deployment,  and  race/ethnicity. 

The  methods  used  to  adjust  for  nonresponse  to  the  2015  WGRR  survey  more  closely 
paralleled  methods  in  the  2014  RMWS  than  the  methods  used  for  the  2012  WGRR.  The  RMWS 
methods  began  with  the  identification  of  key  survey  outcome  variables.  For  the  larger  survey  of 
the  active  duty  component  in  2014,  RAND  identified  six  key  survey  outcome  variables:  three 
types  of  sexual  assault  (penetrative,  non-penetrative,  and  attempted  penetrative)  and  three  types 
of  Military  Equal  Opportunity  (MEO)  violations  (sexually  hostile  work  environment,  sexual  quid 
pro  quo,  and  gender  discrimination).  Because  of  the  smaller  sample  size  of  the  2014  Reserve 
component  sample,  however,  RAND  focused  on  just  three  outcome  variables:  Sexual 
harassment,  gender  discrimination,  and  sexual  assault. 

The  2014  RMWS  nonresponse  adjustment  involved  two  steps,  each  of  which  produced  a 
set  of  models.  For  both  Reserve  and  active  duty  sample  populations,  the  first  step  used  data  from 
the  eligible  respondents  with  completed  responses  to  develop  models  for  the  key  outcome 
variables.  The  models  were  fitted  separately  by  gender.  For  Reserve  members  of  each  gender, 
the  three  outcomes  were  modeled  as  a  function  of  an  extensive  set  of  administrative  variables 
available  for  both  respondents  and  nonrespondents,  resulting  in  six  separate  models  in  total. 
Predicted  values  or  combination  variables  (Morral,  2015)  were  computed  for  both  respondents 
and  nonrespondents,  and  then  these  combination  variables  were  used  in  a  second  model  for  the 
probability  of  response,  along  with  a  limited  number  of  other  predictors:  gender,  reserve 
component,  paygrade,  and  survey  form  type  (paper  vs.  web).  The  reciprocals  of  the  predicted 
values  from  the  second  model  were  used  as  nonresponse  adjustments  and  applied  to  the 
respondents. 

The  approach  to  weighting  used  in  the  2014  RMWS  incorporated  two  significant 
innovations.  First,  a  specific  form  of  machine  learning  model,  generalized  boosted  regression 
(GBM),  was  used  in  place  of  the  logistic  regression  model  used  in  the  2012  WGRR.  In  general, 
the  GBM  model  adapts  more  readily  to  complex  relationships  among  the  dependent  variable  and 
candidate  independent  variables.  Second,  previous  work  of  Little  and  Vartivarian  (2004)  guided 
the  2014  RMWS  approach.  Little  and  Vartivarian  argued  only  information  related  to  key  survey 
outcomes  should  be  included  in  a  nonresponse  model,  otherwise  additional  information  will  only 
increase  the  variance  without  reducing  bias  for  the  key  outcomes.  The  2014  RMWS  used  GBM 
to  summarize  the  relationship  between  the  extensive  auxiliary  information  and  each  of  the  key 
outcome  variables  in  the  form  of  the  predicted  values  from  the  GBM  fit.  The  nonresponse 
adjustment  based  on  the  combination  variables  and  a  limited  number  of  other  characteristics 
could  be  then  expected  to  reduce  nonresponse  bias  while  limiting  the  increase  in  sampling 
variance. 

Preliminary  analyses  published  by  RAND  (Morral,  Gore,  &  Schell,  2014,  2015) 
suggested  advantages  to  the  nonresponse  approach  used  in  the  2014  RMWS,  although  their 
evidence  was  based  primarily  on  the  active  duty  results  the  survey.  On  this  basis,  nonresponse 
adjustment  for  the  2015  WGRR  adopted  these  methods,  but  modifications  were  necessary.  Using 
completed  2015  cases,  six  outcome  variables  were  modeled  for  females:  sexual  harassment, 
gender  discrimination,  sexual  quid  pro  quo,  sexual  assault,  non-penetrative  sexual  assault,  and 
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penetrative  sexual  assault.  For  males,  only  sexual  harassment,  gender  discrimination,  and  sexual 
assault  were  modeled  because  few  incidents  were  reported  for  the  other  three  variables.  Table  5 
provides  a  list  of  the  candidate  auxiliary  variables  considered  for  the  GBM  models.  Appendix  B 
provides  a  more  detailed  version  of  the  table  identifying  the  levels  for  each  categorical  variable 
in  Table  5. 

Unlike  the  2014  RMWS,  the  survey  protocol  of  the  2015  WGRR  excluded  sample 
respondents  who  were  no  longer  Reserve  component  members.  Consequently,  the  2015  WGRR 
paralleled  the  2012  WGRR  in  the  division  of  nonrespondents  by  eligibility  and  completion  of  the 
survey.  The  first  step  of  modeling  nonresponse  in  the  2015  WGRR  followed  the  general  plan  of 
the  2014  RMWS,  creating  combination  variables  corresponding  to  each  key  variable.  The  second 
step  in  the  2014  RMWS  of  modeling  the  propensity  to  respond  became  two  steps  in  the  2015 
WGRR'.  (1)  modeling  known  eligibility  status  to  derive  a  nonresponse  adjustment  for  known 
eligibility  status,  and  (2)  fitting  a  model  to  eligible  cases  to  determine  the  probability  of 
completing  the  survey  in  order  to  derive  a  second  nonresponse  adjustment.  Both  the  eligibility 
and  completion  models  incorporated  the  combination  variables  as  well  as  auxiliary  variables  that 
are  used  in  the  raking  adjustments  that  follow:  paygrade,  reserve  program,  race/ethnicity,  and 
component.  Both  sets  of  models  were  fitted  separately  by  gender.  Like  the  2012  WGRR 
analysis,  the  GBM  models  were  weighted;  the  first  by  the  sampling  weight,  and  the  second  by 
the  eligibility-adjusted  weight  resulting  from  multiplying  the  sampling  weight  by  the  eligibility 
status  adjustment. 

To  further  detail  the  nonresponse  adjustments  used  in  the  2015  WGRR ,  in  Table  3,  case 
dispositions  2,  3,  4,  and  5  denote  cases  with  known  eligibility,  whereas  case  dispositions  8,  9,  10, 
and  1 1  correspond  to  cases  for  which  eligibility  is  unknown.  Consequently,  the  first  of  the  two 
nonresponse  adjustments  increased  the  weights  for  case  dispositions  2,  3,  4,  and  5  to  represent 
dispositions  8,  9,  10,  and  11.  The  second  adjustment  increased  the  weights  of  complete  cases 
with  disposition  4  to  compensate  for  incomplete  eligible  cases  with  disposition  5. 

To  increase  response  to  the  2015  WGRR ,  nonrespondents  to  the  web  version  of  the 
survey  were  sent  a  paper  form  of  the  questionnaire.  The  paper  version  included  the  key  survey 
items,  but  it  omitted  many  secondary  items  on  the  web  questionnaire,  presenting  the  recipient 
with  approximately  100  questions  instead  of  the  approximately  230  on  the  web  version.  The 
primary  set  of  weights  was  based  on  responses  from  the  full  data  set  including  both  the  web  and 
paper  versions.  To  support  analysis  of  items  only  on  the  web  version,  a  second  set  of  weights 
was  produced,  following  the  same  steps  as  the  full  data  set  including  the  paper  questionnaire. 

For  this  weighting,  all  paper  questionnaire  respondents  were  treated  as  nonrespondents, 
including  in  the  fitting  of  the  GBM  models.  This  second  set  of  weights  is  intended  solely  for 
analysis  of  web-only  items.  The  primary  set  of  weights  provides  the  basis  for  estimating  the  key 
outcomes  from  the  survey  items  collected  on  both  the  web  and  paper  versions  of  the 
questionnaire. 
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Table  5. 

Variables  Used  for  the  Eligibility  and  Completion  Adjustments 


Demographic  Factors 

•  AFQT  Score  Percentile 

•  Age  as  of  August  2015 

•  Family  Status 

•  Education 

•  Race/Ethnic  Category 

•  US  Citizenship  Origin  Code 

•  US  Citizenship  Status  Code 
Military  Career  Factors 

•  Military  Accession  Program 

•  Active  Duty  Status 

•  Active  Duty  and  Special  Operations  Status 

•  Active  Duty  Begin  Date 

•  Active  Duty  End  Date 

•  Number  of  Occurrences  on  Active  Duty  from  July 
2014  through  July  2015 

•  Number  of  Days  Activated  from  July  2014  through 
July  2015 

•  Active  Guard  &  Reserve,  or  Full  Time  National 
Guard  Duty  Statue  ID 

•  Combat  Occupation  Flag 

•  Current  Deployment  Status 

•  Number  of  Deployments 

•  Deployment  Flag  in  the  Last  12  months 

•  Deployment  Flag  in  the  Last  24  months 

•  Number  of  Months  Deployed  since  9/1 1/2011 

•  Number  of  Months  Deployed  from  August  2014 
through  August  2015 

•  Duty  DoD  Occupation  Code 

•  Paygrade 

•  Primary  Occupation  Code 

•  DoD  Primary  Occupation  Area  Code1 

•  Primary  Regular  Component  Service  Indicator 

•  Selected  Reserve  Obligated  Service  Projected  End 
Date 

•  Date  of  First  Affiliation  or  Enlistment  in  Reserve 
Component 

•  Date  of  First  Appointment,  Enlistment,  or 
Conscription  into  a  Uniformed  Service  of  the  US 

•  Eligibility  Status  as  of  August  2015 


Military  Career  Factors  (continued) 

•  Years  of  service 

•  Reserve  Category  Programs 

•  Reserve  Organization  Code 

•  Reserve  Category  Group  Code 

•  Reserve  Subcategory  Code 

•  Reserve  Category  Code 

•  Reserve  Component  Category  Code 
Military  Environment  Factors 

•  Number  of  Members1  in  Member’s  Assigned  UIC 

•  Number  of  Males1  in  Member’s  Assigned  UIC 

•  Number  of  Members1  in  Member’s  Duty  UIC 

•  Number  of  Males2  in  Member’s  Duty  UIC 

•  Number  of  Members1  in  Member’s  Primary 
Occupation’ 

•  Number  of  Males2  in  Member’s  Primary  Occupation  ’ 

•  Number  of  Members'  in  Member’s  Area  Occupation  ’ 

•  Number  of  Males2  in  Member’s  Area  Occupation 

•  Percent  of  Males2  in  Member’s  Assigned  UIC. 

•  Percent  of  Males2  in  Member’s  Duty  UIC. 

•  Percent  of  Males2  in  Member’s  Primary  Occupation1 

•  Percent  of  Males2  in  Member’s  Area  Occupation3 

•  Assigned  UIC  Change  of  Station  Flag  as  of  March 

2014 

•  Assigned  UIC  Change  of  Station  Flag  as  of  August 

2015 

•  Duty  UIC  Change  of  Station  Flag  as  of  March  2014 

•  Duty  UIC  Change  of  Station  Flag  as  of  August  2015 
Survey  Fielding  Factors 

•  Invalid  Army  Email  Address  Flag 

•  Email  Address  Purchase  Flag 

•  Email  Address  Flag 

•  Number  of  Email  Addresses 

•  First  Letter  Returned  as  PND 

•  Mail  Address  Flag 

•  Change  in  Mailing  Address  since  Sample  Frame 
Development 

•  Home  Address  Flag 


1  Reserve  members 

2  Reserve  males 

3  Collapsed  primary  occupation  (first  2  digits  of  primary  occupation  code) 


The  nonresponse-adjusted  weights  were  then  modified  through  a  process  called  raking. 
The  purpose  of  raking  is  to  use  known  infonnation  about  the  survey  population  to  increase  the 
precision  of  population  estimates.  This  information  consists  of  totals  for  different  levels  of 
variables  (such  as  demographic  characteristics).  For  example,  the  variable  of  gender  has  two 
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levels:  male  and  female.  During  the  raking  process,  sampled  individuals  are  first  categorized 
into  the  cells  of  a  table  defined  by  two  or  more  variables — called  raking  dimensions.  The  goal  of 
raking  is  to  adjust  the  weights  so  that  they  add  up  to  the  known  totals — called  control  totals — for 
the  different  levels  within  each  raking  dimension.  Proceeding  one  dimension  at  a  time,  raking 
computes  a  proportional  adjustment  to  the  weights  associated  with  each  level  of  the  raking 
dimension.1  After  all  dimensions  are  adjusted,  the  process  is  repeated  until  the  totals  for  all 
levels  of  the  raking  dimensions  are  equal  to  the  corresponding  control  totals  (at  least  within  a 
specified  tolerance). 

Control  totals  were  computed  from  information  from  the  sampling  frame.  There  were 
five  raking  dimensions,  defined  by  gender,  pay-grade  groupings,  Reserve  component,  Reserve 
program,  and  race/ethnicity,  as  follows: 

•  Paygrade  groupings  (7  levels) 

•  Gender  (2  levels)  by  pay-grade  groupings  (5  levels) 

•  Gender  (2  levels)  by  Reserve  program  (4  levels) 

•  Gender  (2  levels)  by  race/ethnicity  (2  levels) 

•  Gender  (2  levels)  by  Reserve  component  (6  levels)  by  pay-grade  groupings  (2  levels) 

Table  6  provides  additional  details  about  the  levels  of  the  variables  used  to  define  the  five 
raking  dimensions. 


1  Raking  is  so  named  because  it  is  analogous  to  smoothing  the  soil  in  a  garden  plot  by  alternately  working  soil  back 
and  forth  with  a  rake  in  two  perpendicular  directions. 
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Table  6. 

Description  of  Raking  Dimensions 


Dimension  # 

Variable  1 

Variable  2 

Variable  3 

Description  Categories 

Description  Categories 

Description  Categories 

1 

Paygrade  E1-E3, 

groupings  E4, 

E5-E6, 

E7-E9, 

W1-W5, 

01-03, 

04-06 

2 

Gender  Female, 

Male 

Paygrade  E1-E4, 

groupings  E5-E9, 

W1-E5, 

01-03, 

04-06, 

3 

Gender  Female, 

Male 

Reserve  TPU, 

program  AGR, 

MilTech, 

IMA 

4 

Gender  Female, 

Male 

Race/ethnicity  Non-PIispanic 
White, 

Total  Minority 

5 

Gender  Female, 

Male 

Reserve  ARNG, 

component  ANG, 

USAR, 

USNR, 

USMCR, 

USAFR 

Paygrade  Enlisted, 

groupings  Officers 

Table  7  summarizes  the  distributions  of  the  sampling  weights,  intermediate  weights,  final 
weights,  and  corresponding  adjustment  factors  by  eligibility  status  for  the  primary  weights. 
Eligible  respondents  are  those  individuals  who  were  not  only  eligible  to  participate  in  the  survey 
but  also  completed  at  least  one  of  the  critical  sexual  assault  questions.  Record  ineligible 
individuals  are  those  who  were  not  eligible  to  participate  in  the  survey  according  to 
administrative  records;  no  weights  were  computed  for  these  cases.  Table  7  also  indicates  the 
mean  of  the  sampling  weights,  intermediate  weights,  and  final  weights  by  eligibility  status.  Two 
tables  in  Appendix  C  and  Appendix  D  show  summary  of  weights  by  gender. 

The  sampling  weights,  which  are  the  reciprocals  of  the  probability  of  selection  into  the 
sample,  take  the  value  1  for  the  census  of  females,  who  were  all  selected  for  the  study.  The  mean 
of  the  sampling  weights  for  males  is  2.32.  The  nonresponse  adjustment  for  eligibility  status  that 
follows  next  makes  the  biggest  single  adjustment  to  the  weights,  in  terms  of  increasing  both  the 
mean  and  the  coefficient  of  variation  (c.v.)  of  the  weights.  The  two  remaining  adjustments  for 
nonresponse  among  the  eligible  population  and  the  final  raking  have  a  modest  effect  on 
increasing  the  mean  weight.  The  corresponding  factors  shown  in  the  last  two  columns  of  Table  7 
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have  small  c.v.’s;  in  other  words,  the  factors  in  each  column  differ  from  each  other  by  relatively 
small  amounts. 


Table  7. 

Distribution  of  Weights  and  Adjustment  Factors  by  Eligibility  Status 


Eligibility 

Status 

Statistic 

Sampling 

Weight 

Eligibility 

Status 

Adjusted 

Weight 

Complete 

Eligible 

Response 

Adjusted 

Weight 

Final  Weight 
With 

Nonresponse 
and  Raking 
Factors 

Eligibility 

Status 

Factor 

Complete 

Eligible 

Response 

Factor 

Raking 

Factor 

Eligible 

Respondents 

N 

87,127 

87,127 

87,127 

87,127 

87,127 

87,127 

87,127 

MIN 

1.00 

1.14 

1.17 

1.18 

1.07 

1.01 

0.77 

MAX 

5.69 

121.51 

125.96 

135.92 

64.72 

1.12 

1.35 

MEAN 

1.79 

8.55 

8.74 

9.10 

5.05 

1.02 

1.03 

STD 

0.96 

8.17 

8.36 

8.95 

4.64 

0.01 

0.08 

CV 

0.53 

0.96 

0.96 

0.98 

0.92 

0.01 

0.08 

Eligible, 

incomplete 

Response 

N 

1,985 

1,985 

1,985 

1,985 

1,985 

1,985 

1,985 

MIN 

1.00 

1.18 

0.00 

0.00 

1.18 

0.00 

MAX 

5.69 

57.32 

0.00 

0.00 

30.53 

0.00 

MEAN 

1.65 

8.48 

0.00 

0.00 

5.31 

0.00 

STD 

0.92 

8.35 

0.00 

0.00 

4.76 

0.00 

CV 

0.56 

0.98 

0.90 

Self/Proxy 

Ineligibles 

N 

1,541 

1,541 

1,541 

1,541 

1,541 

1,541 

1,541 

MIN 

1.00 

1.43 

1.43 

1.48 

1.40 

1.00 

0.77 

MAX 

5.69 

114.73 

114.73 

123.58 

64.72 

1.00 

1.34 

MEAN 

1.74 

15.06 

15.06 

15.89 

9.07 

1.00 

1.05 

STD 

0.80 

14.62 

14.62 

15.57 

8.34 

0.00 

0.08 

CV 

0.46 

0.97 

0.97 

0.98 

0.92 

0.00 

0.08 

Nonrespondents 

N 

384,491 

384,491 

384,491 

384,491 

384,491 

384,491 

384,491 

MIN 

1.00 

0.00 

0.00 

0.00 

0.00 

MAX 

5.69 

0.00 

0.00 

0.00 

0.00 

MEAN 

1.66 

0.00 

0.00 

0.00 

0.00 

STD 

0.65 

0.00 

0.00 

0.00 

0.00 

CV 

0.39 

Record 

Ineligibles 

N 

10,630 

10,630 

10,630 

10,630 

10,630 

10,630 

10,630 

MIN 

1.00 

1.00 

1.00 

0.00 

MAX 

5.69 

5.69 

5.69 

0.00 

MEAN 

1.59 

1.59 

1.59 

0.00 

STD 

0.58 

0.58 

0.58 

0.00 

CV 

0.36 

0.36 

0.36 
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Under  simplifying  assumptions  that  were  applied  based  on  Kish  (Kish,  1965)  which 
approximates  the  relative  increase  due  to  weight  variation  as  approximately  1  plus  the  c.v.  of  the 
weights  squared.  Because  the  coefficient  of  variation  of  the  weights  is  less  than  1,  especially 
when  analyzed  for  females,  0.84,  and  males,  0.81,  separately,  the  increase  in  variance  due  to 
weighting  is  less  than  a  factor  of  2.  Given  the  task  of  the  weighting  adjustments  is  to 
compensate  for  differential  nonresponse  and  its  possible  impact  on  the  bias  of  key  outcome 
variables,  the  outcomes  shown  in  Table  7  appear  reasonable. 

Table  8  exhibits  the  sum  of  the  weights  at  different  stages  of  weighting.  The  weights 
adjusted  for  known  eligibility  status  distribute  the  sampling  weights  for  nonrespondents  with 
unknown  eligibility  status  among  the  remaining  dispositions.  The  eligible  response  adjusted 
weights  then  compensate  for  eligible  respondents  providing  incomplete  surveys.  By  design,  the 
final  raking  adjustments  redistribute  record  ineligibles  and  other  dispositions  excluded  from  the 
final  weights  to  match  total  number  in  the  original  frame. 


Table  8. 

Sum  of  Weights  by  Eligibility  Status 


Eligibility  Category 

Sum  of  Sampling 
Weights 

Sum  of  Eligibility 
Status  Adjusted 
Weights 

Sum  of  Complete 
Eligible  Response 
Adjusted  Weights 

Sum  of  Final 
Weights  With 
Nonresponse  and 
Raking 
Adjustments 

Eligible  Respondents 

156,111 

744,723 

761,538 

792,528 

Eligible,  Incomplete 
Response 

3,273 

16,841 

0 

0 

Self/proxy  Ineligibles 

2,677 

23,208 

23,208 

24,479 

Nonrespondents 

638,042 

0 

0 

0 

Record  ineligibles 

16,904 

16,904 

16,904 

0 

Total 

817,007 

801,676 

801,650 

817,007 

Comparison  to  the  2014  RAND  Military  Workplace  Study 

RAND  found  that  increasing  the  number  of  weighting  variables  and  using  GBM 
improved  the  2014  RMWS  survey  weights,  therefore,  DMDC  decided  to  also  use  this  approach 
for  the  2015  WGRR.  The  description  of  the  2015  WGRR  weighting  was  set  in  the  context  of  the 
2014  RMWS  in  the  preceding  section.  The  comparison  is  further  elaborated  here. 

The  software  used  for  the  2015  WGRR  was  built  on  the  approach  used  in  2014  RMWS. 
Both  weightings  used  the  statistical  computing  software  R  and  specifically  functions  from  the 
packages  gbm  (Ridgeway,  2009)  and  twang  (Ridgeway,  2004).  RAND  researchers  provided  the 
specific  R  scripts  they  used  for  their  final  production  runs  of  the  2014  RMWS  weighting. 
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The  weighting  for  the  2015  WGRR  also  differed  in  some  respects  from  the  2014  RMWS. 
The  2015  WGRR  weighting  incorporated  the  two  nonresponse  steps  used  in  the  2012  WGRR, 
necessitating  use  of  weights  throughout  the  analysis.  Some  of  the  modeling  in  the  2014  RMWS 
had  been  unweighted.  In  2015  WGRR ,  the  nonresponse  models  were  separated  by  gender  at  both 
the  initial  stage  of  creating  the  combined  variables  and  at  the  second  stage  of  creating  the 
nonresponse  weights,  whereas  both  genders  had  been  combined  in  the  second  stage  in  the  2014 
RMWS. 

Variance  Estimation 

Sampling  error  is  the  uncertainty  associated  with  an  estimate  that  is  based  on  data 
gathered  from  a  sample  of  the  population  rather  than  the  full  population.  Note  that  sample-based 
estimates  will  vary  depending  on  the  particular  sample  selected  from  the  population.  Measures 
of  the  magnitude  of  sampling  error,  such  as  the  variance  and  the  standard  error  (the  square  root 
of  the  variance),  reflect  the  variation  in  the  estimates  over  all  possible  samples  that  could  have 
been  selected  from  the  population  using  the  same  sampling  methodology.  Analysis  of  the  2015 
WGRR  data  required  a  variance  estimation  procedure  that  accounted  for  the  weighting 
procedures.  The  final  step  of  the  weighting  process  was  to  define  strata  for  variance  estimation 
by  Taylor  series  linearization.  The  2015  WGRR  variance  estimation  strata  correspond  exactly 
with  the  128  strata  used  to  select  the  sample.  For  each  strata/variance  strata,  DMDC  ensured  that 
there  were  at  least  30  complete  eligible  responses  with  non-zero  final  weights. 

Multiple  Comparison  Adjustment 

When  statistically  comparing  groups  (e.g.,  Army  vs.  Navy  estimates  of  the  effectiveness 
of  the  training),  a  statistical  hypothesis  whether  there  are  no  differences  (null  hypothesis)  versus 
there  are  differences  (alternative  hypothesis)  is  tested.  DMDC  mainly  uses  independent  two 
sample  t-tests  for  its  statistical  tests.  The  conclusions  are  usually  based  on  the  p-value  associated 
with  the  test-statistic.  If  the  p-value  is  less  than  the  critical  value  then  the  null  hypothesis  is 
rejected.  Any  time  a  null  hypothesis  is  rejected  (a  conclusion  that  estimates  are  significantly 
different),  it  is  possible  this  conclusion  is  incorrect.  In  reality,  the  null  hypothesis  may  have  been 
true,  and  the  significant  result  may  have  been  due  to  chance.  A  p-value  of  0.05  means  there  is  a 
five  percent  chance  of  finding  a  difference  as  large  as  the  observed  result  if  the  null  hypothesis 
were  true. 

In  survey  research  there  is  often  interest  in  conducting  multiple  comparisons.  For 
example,  1)  testing  whether  the  percentage  of  sexual  assaults  among  Army  Reserve  is  the  same 
as  the  percentage  of  sexual  assaults  across  all  other  components,  and  2)  testing  that  the 
percentage  of  sexual  harassments  for  Navy  Reserve  is  the  same  as  the  percentage  of  sexual 
harassments  with  all  other  components  and  so  on.  When  performing  multiple  independent 
comparisons  on  the  same  data  the  question  becomes:  “Does  the  interpretation  of  the  p-value  for  a 
single  statistical  test  hold  for  multiple  comparisons?”  If  200  independent  statistical 
(significance)  tests  were  conducted  at  the  0.05  significance  level,  and  the  null  hypothesis  is 
supported  for  all,  10  of  the  tests  would  be  expected  to  be  significant  at  the  p-value  <  0.05  level 
simply  due  to  chance.  These  10  tests  would  have  incorrectly  assumed  to  be  statistically 
significant — known  as  false  positives  or  false  discoveries.  When  a  single  significance  test  is 
conducted,  the  error  rate — the  probability  of  false  discoveries — is  the  p-value  itself.  When  more 
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than  one  significance  test  is  conducted,  the  probability  of  false  discoveries  and  the  number  of 
false  discoveries  increases,  i.e.,  the  more  tests  that  are  conducted  the  greater  the  number  of  false 
discoveries. 

This  is  known  in  statistical  hypothesis  testing  as  the  multiple  comparisons  problem. 
Therefore,  it  is  important  to  control  the  false  discoveries  when  performing  multiple  independent 
tests  to  reach  more  accurate  conclusions.  Numerous  techniques  have  been  developed  to  control 
the  false  positive  error  rate  associated  with  conducting  multiple  statistical  tests  (multiple 
comparisons)  and  there  is  no  universally  accepted  approach  for  dealing  with  it. 

The  method  that  DMDC  uses  to  control  for  false  discoveries  is  known  as  the  False 
Discovery  Rate  correction  (FDR)  developed  by  Benjamini  and  Hochberg  (1995).  FDR  is 
defined  as  the  expected  percentage  of  erroneous  rejections  among  all  rejections.  The  goal  is  to 
control  the  false  discovery  rate  which  is  the  proportion  of  "discoveries"  (significant  results)  that 
are  actually  false  positives.  The  approach  can  be  summarized  as  follows: 

•  Determine  the  number  of  comparisons  (tests)  of  interest,  call  it  m; 

•  Determine  the  tolerable  False  Discovery  Rate  (FDR  Rate),  call  it  a; 

•  Calculate  the  p-value  for  each  statistical  test; 

•  Sort  the  individual  p-values  from  smallest  to  largest  and  rank  them,  call  the  rank  k. 

•  For  each  ranked  p-value  calculate  the  FDR-adjusted  alpha  (threshold)  which  is 
defined  as  (k  *  oc  )/m 

•  Determine  the  cutoff  delineating  statistically  significant  results  from  non-significant 
results  in  the  sorted  file  as  follows:  Look  for  the  maximum  rank  (k)  such  that  the 
ordered  p-value  is  less  than  the  FDR-adjusted  alpha  (i.e.,  look  for  the  maximum  k 
after  which  the  p-value  becomes  greater  than  the  threshold),  call  this  maximum  k  the 
cutoff.  Any  comparison  (p-value)  with  rank  less  than  the  cutoff  is  considered 
statistically  significant. 

DMDC  computed  the  FDR  thresholds  (FDR  adjusted  alpha)  separately  for  the  two  types 
of  comparisons — current  year  and  trends.  For  both  types  of  tests,  DMDC  implemented  FDR 
Multiple  Comparison  corrections  to  control  the  expected  rate  of  false  discoveries  (Type  I  errors) 
at  oc  =  0.05.  For  the  current  year  estimates  from  the  2015  WGRR,  DMDC  performed  59,724 
separate  statistical  tests  (e.g.,  testing  whether  the  sexual  assault  rate  among  Army  Reserve  is  the 
same  as  the  sexual  assault  rate  across  all  other  components).  Of  the  59,724  current  year 
statistical  tests,  19,165  were  statistically  significant.  In  addition,  DMDC  performed  another  180 
separate  statistical  tests  for  trends  to  compare  estimates  from  the  2015  WGRR  to  the  2014 
RMWS.  For  trends,  27  of  the  180  statistical  tests  were  significant. 
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Location,  Completion,  and  Response  Rates 

Location,  completion,  and  response  rates  were  calculated  in  accordance  with  the 
recommendations  of  the  American  Association  for  Public  Opinion  Research  (AAPOR,  2015 
Standard  Definitions),  which  estimates  the  proportion  of  eligible  respondents  among  cases  of 
unknown  eligibility. 

The  location  rate  (LR)  uses  AAPOR  standard  formula  CON2  and  is  defined  as 

(I  +  P)  +  (R  +  NC  +  O )  adjusted  located  sample  N L 

L  —  —  —  . 

(I  +  P)+  ( R  +  NC  +  0)  +  e(UO )  adjusted  eligible  sample  N E 

The  completion  rate  (CR)  uses  AAPOR  standard  formula  COMR  and  is  defined  as 
(I  +  P)  usable  responses  NR 

C/f  V  —  —  . 

(/  +  P)  +  (R  +  NC  +  O )  adjusted  located  sample  N , 


The  response  rate  (RR)  uses  AAPOR  standard  fonnula  RR4  and  is  defined  as 


RR 


_ (I  +  P) _ 

(/  +  R)+(R  +  NC  +  O)  +  e(UO) 


usable  responses  _  NR 
adjusted  eligible  sample  NE 


Where 


/  =  Fully  complete  responses  according  to  RR4  (>  80%  complete) 

P  =  Partially  complete  responses  according  to  RR4  (50  -  80%  complete) 

R  =  Refusal  and  break-off  according  to  RR4  (<  50%  complete) 

NC  =  Non-contact 
O  =  Other 

e(ZJO)  =  Estimated  eligibility  of  cases  unknown 
Nl  =  Adjusted  located  sample 
Ne  =  Adjusted  eligible  sample 
Nr  =  Usable  responses 

Table  9  shows  the  corresponding  sample  disposition  codes  associated  with  the  response 
categories. 
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Table  9. 

Disposition  Codes  for  Response  Rates 


Response  Category 

SAMP_DC  Values 

Eligible  Sample 

4,  5,  8,  9,  10,  11 

Located  Sample 

4,5,  8,  9,  11 

Usable  Response 

4 

Not  Returned 

11 

Eligibility  Determined 

2,  3,  4,  5,  8,  9 

Self-Report  Ineligible 

2,3 

Ineligibility  Rate 

The  ineligibility  rate  (IR)  is  defined  as  the  following  and  needs  to  be  calculated  for  both 
weighted  and  unweighted  to  be  applied  to  Table  9: 

IR  =  Self  Report  Ineligible/Eligibility  Determined. 

Estimated  Ineligible  Postal  Non-Deliverable/Not  Located  Rate 

The  estimated  ineligible  postal  non-deliverable  or  not  located  (IPNDR)  is  defined  as: 
IPNDR  =  (Eligible  Sample  -  Located  Sample)  *  IR. 

Estimated  Ineligible  Nonresponse 

The  estimated  ineligible  nonresponse  (EINR)  is  defined  as: 

EINR  =  (Not  Returned)  *  IR. 

Adjusted  Location  Rate 

The  adjusted  location  rate  (ALR)  is  defined  as: 

ALR  =  (Located  Sample  -  EINR)/(Eligible  Sample  -  IPNDR  -  EINR). 

Adjusted  Completion  Rate 

The  adjusted  completion  rate  (ACR)  is  defined  as: 

ACR  =  (Eligible  Response)/(Located  Sample  -  EINR). 
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Adjusted  Response  Rate 


The  adjusted  response  rate  (ARR)  is  defined  as: 

ARR  =  (Eligible  Response)/(Eligible  Sample  -  IPNDR  -  EINR). 

Table  10  shows  the  weighted  sample  counts  used  to  compute  the  overall  response  rates. 

The  final  response  rate  is  the  product  of  the  location  rate  and  the  completion  rate.  Table 
1 1  shows  both  weighted  and  unweighted  location,  completion,  and  response  rates  for  the  2015 
WGRR. 


Finally,  Table  12  shows  weighted  location,  completion,  and  response  rates  for  the  full 
sample  by  the  stratification  variables.  The  final  weighted  response  rate  for  the  survey  was  19.8 
percent  which  rounds  to  20.0  percent. 


Table  10. 

Comparison  of  the  Final  Weighted  Respondents  Relative  to  the  Drawn  Sample 


Case  Disposition  Categories 

Sample  Counts 

Weighted  Estimates 

Sample 

Size 

Percent 

Estimated 

Total 

Percent 

Drawn  sample  and  population 

485,774 

100% 

817,007 

100% 

Ineligible  on  master  files 

-10,630 

2.2% 

-16,904 

2.1% 

Self-reported  ineligible 

-1,541 

0.3% 

-2,677 

0.3% 

Total:  Ineligible 

-12,171 

2.5% 

-19,581 

2.4% 

Eligible  sample 

473,603 

97.5% 

797,427 

97.6% 

Not  located  (estimated  ineligible) 

-777 

0.2% 

-1,201 

0.1% 

Not  located  (estimated  eligible) 

-45,815 

9.4% 

-72,978 

8.9% 

Total  not  located 

-46,592 

9.6% 

-74,179 

9.1% 

Located  sample 

427,011 

87.9% 

723,248 

88.5% 

Requested  removal  from  survey  mailings 

-1,176 

0.2% 

-2,248 

0.3% 

Returned  blank 

-611 

0.1% 

-1,081 

0.1% 

Skipped  key  questions 

-1,985 

0.4% 

-3,273 

0.4% 

Did  not  return  a  survey  (estimated  ineligible) 

-5,603 

1.2% 

-9,072 

1.1% 

Did  not  return  a  survey  (estimated  eligible) 

-330,509 

68.0% 

-551,463 

67.5% 

Total:  Nonresponse 

-339,884 

70.0% 

-567,137 

69.4% 

Eligible  responses 

87,127 

17.9% 

156,111 

19.1% 
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Table  11. 

Location,  Completion,  and  Response  Rates 


Type  of  Rate 

Computation 

Unweighted 

Weighted 

Location 

Adjusted  located  sample/Adjusted  eligible  sample 

90.2% 

90.7% 

Completion 

Usable  responses/Adjusted  located  sample 

20.7% 

21.9% 

Response 

Usable  responses/Adjusted  eligible  sample 

18.6% 

19.8% 

Table  12. 

Rates  for  Full  Sample  and  Stratification  Level  of  Variable 


Domain 

Variable 

Domain 

Sample 

Size 

Eligible 

Responses 

Sum  of 
Weights 

Location 

Rate 

Completion 

Rate 

Response 

Rate 

Sample 

Full  Sample 

485,774 

87,127 

817,007 

90.7% 

21.9% 

19.8% 

Reserve 

Component 

(RORGCD) 

Army  National  Guard 

186,481 

25,172 

348,599 

89.7% 

18.0% 

16.1% 

US  Army  Reserve 

121,036 

18,674 

197,698 

90.3% 

19.1% 

17.2% 

US  Navy  Reserve 

36,245 

8,053 

58,227 

85.9% 

28.1% 

24.1% 

US  Marine  Corps 

Reserve 

36,364 

4,002 

38,468 

88.6% 

13.7% 

12.2% 

Air  National  Guard 

61,695 

19,195 

104,818 

96.0% 

33.1% 

31.8% 

US  Air  Force  Reserve 

43,953 

12,031 

69,197 

94.5% 

29.8% 

28.2% 

Gender 

(RSEX2) 

Male 

331,332 

52,421 

662,565 

90.8% 

21.0% 

19.0% 

Female 

154,442 

34,706 

154,442 

90.4% 

25.8% 

23.3% 

Paygrade 

Grouping 

(RPAYGRP9) 

E1-E4 

238,102 

18,575 

352,772 

86.6% 

9.0% 

7.8% 

E5-E9 

179,140 

45,178 

336,347 

93.2% 

28.4% 

26.5% 

W1-W5 

5,773 

2,203 

12,187 

97.0% 

40.6% 

39.4% 

01-03 

33,684 

9,125 

59,530 

93.2% 

29.0% 

27.0% 

04-06 

29,075 

12,046 

56,171 

97.5% 

44.4% 

43.3% 

Reserve 

Program 

(RPROG1) 

TPU 

414,431 

57,902 

666,695 

89.7% 

16.4% 

14.7% 

AGR 

33,432 

13,773 

76,747 

93.9% 

45.5% 

42.7% 

MilTech 

29,273 

12,737 

61,484 

97.1% 

44.6% 

43.4% 

IMA 

8,638 

2,715 

12,082 

96.6% 

34.5% 

33.3% 

Nonresponse  Bias  Analysis 

Survey  nonresponse  has  the  potential  to  introduce  bias  in  the  estimates  of  key  outcomes. 
To  the  extent  that  nonrespondents  and  respondents  differ  on  observed  characteristics,  DMDC  can 
use  weights  to  adjust  the  sample  so  the  weighted  respondents  match  the  full  population  on  the 
most  critical  characteristics.  This  eliminates  the  portion  of  nonresponse  bias  (NRB)  associated 
with  those  observed  variables  if  these  variables  are  strongly  associated  with  the  behaviors. 

When  all  NRB  can  be  eliminated  in  this  manner,  the  missingness  is  called  ignorable  or  missing 
at  random  (Little  &  Rubin,  2002).  The  more  observable  demographic  variables  that  were 
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incorporated  into  the  weights,  the  more  plausible  it  is  to  assume  that  the  weights  eliminate  any 
NRB. 


The  objective  of  this  research  was  to  assess  the  extent  of  NRB  for  the  estimated 
percentage  of  sexual  assaults  (henceforth  this  rate  will  be  referred  to  as  sexual  assault)  in  the 
Reserve  components.  The  purpose  of  the  percentage  of  sexual  assaults  was  to  provide  the  policy 
offices  and  the  Department  with  an  overall  estimate  of  Reserve  component  members  who 
experienced  sexual  assault  in  the  last  12  months.  The  level  of  nonresponse  bias  can  vary  for 
every  question  on  the  survey,  but  DMDC  focused  on  the  sexual  assault  rate  because  this  tended 
to  be  one  of  the  more  central  questions  on  the  survey.  Nonresponse  bias  occurs  when  survey 
respondents  are  systematically  different  from  nonrespondents.  Statistically,  the  bias  in  a 
respondent  mean  (e.g.,  sexual  assault  rate)  is  a  function  of  the  response  rate  and  the  relationship 
(covariance)  between  response  propensities  and  the  estimated  statistics  (i.e.,  sexual  assault  rate), 
and  takes  the  following  form: 

Bias  (yr)  =  <?yGp,  where  oyp  =  covariance  between  y  and  response 

propensity,  p 

NRB  can  occur  with  high  or  low  survey  response  rates,  but  the  decrease  in  overall  survey 
response  rates  within  the  Department  as  well  as  civilian  studies  in  the  past  decade  has  resulted  in 
a  greater  focus  on  potential  NRB.  DMDC  investigated  the  presence  of  NRB  using  many 
different  methods,  and  this  report  summarizes  the  following  methods  and  results: 

1 .  Compare  known  population  values  with  weighted  survey  estimates, 

2.  Analyze  item  missing  data  for  sexual  assault  questions, 

3.  Analysis  of  DMDC’ s  survey  of  nonrespondents, 

4.  Evaluate  the  sensitivity  of  different  post-survey  adjustments  (weighting  methods)  on 
survey  estimates, 

Summary  of  Findings 

NRB  is  difficult  to  assess.  Most  authors  recommend  averaging  across  several  different 
studies  to  measure  NRB  (Montaquila  &  Olson,  2012).  DMDC  has  taken  that  approach  here  and 
conducted  four  studies  to  assess  NRB  in  sexual  assault  estimates.  Based  on  these  four  studies, 
DMDC  does  not  find  evidence  of  significant  NRB  in  sexual  assault  estimates  from  the  2015 
WGRR. 

We  summarize  the  results  from  each  study  below: 

1.  Compare  known  population  values  with  weighted  survey  estimates  — DMDC 
compared  weighted  estimates  of  officially  reported  sexual  assaults  from  the  survey 
with  actual  reported  sexual  assaults  to  SAPRO.  The  survey  estimates  are  higher  than 
the  official  reports  (but  within  the  margins  of  error),  and  a  possible  conclusion  is 
sexual  assault  victims  are  more  likely  to  complete  the  2015  WGRR.  However,  this 
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finding  would  contradict  earlier  NRB  studies  on  sexual  assault  where  both  DMDC 
and  RAND  concluded  that  sexual  assault  was  underestimated  (DMDC,  2013  and 
Morral,  2014)  and  should  be  taken  with  caution.  From  this  analysis,  DMDC 
concludes  there  is  little  evidence  of  NRB  in  sexual  assault  estimates  from  the  2015 
WGRR. 

2.  Analyze  item  missing  data  for  sexual  assault  questions — Item  missing  data  rates 
for  the  2015  WGRR  sexual  assault  questions  are  similar  to  missing  data  rates  from 
other  DMDC  surveys.  In  addition,  there  is  no  evidence  that  prior  victims  of  sexual 
assault  are  completing  the  survey  at  different  rates  than  members  who  have  not  been 
previously  assaulted.  From  this  analysis,  DMDC  concludes  there  is  little  evidence  of 
NRB  in  sexual  assault  estimates  from  the  2015  WGRR. 

3.  Analysis  of  DMDC’s  survey  of  nonrespondents — Estimates  from  the  2015  WGRR 
nonresponse  study  ( 2015  WGRR-N)  are  comparable  to  production  estimates.  The 
estimates  for  three  out  of  four  matching  questions  are  within  one  percentage  point  of 
each  other,  and  none  of  the  four  differences  is  statistically  significant.  From  this 
analysis,  DMDC  concludes  there  is  little  evidence  of  NRB  in  sexual  assault  estimates 
from  the  2015  WGRR. 

4.  Evaluate  the  sensitivity  of  different  post-survey  adjustments  (weighting 
methods)  on  survey  estimates — Analysis  of  estimates  using  two  different  weighting 
methods  show  both  the  weights  and  key  survey  estimates  are  robust  to  the  choice  of 
weighting  methods.  From  this  analysis,  DMDC  concludes  there  is  little  evidence  of 
NRB  in  sexual  assault  estimates  from  the  2015  WGRR. 


Section  1:  Compare  Known  Population  Values  with  Weighted  Survey  Estimates 

To  assess  total  survey  error,  one  common  method  is  to  compare  a  known  parameter  to  a 
weighted  estimate  from  the  survey.  If  DMDC  sampling,  measurement,  weighting,  and  analysis 
methods  performed  well,  confidence  interval  of  estimates  should  frequently  contain  the  true 
parameters.  In  this  investigation,  DMDC  examined  the  number  of  reported  sexual  assaults  in  the 
Reserve  component.  A  similar  type  of  analysis  was  perfonned  by  RAND  for  the  2014  RMWS 
(Morral,  2016).  It  is  important  to  point  out  that  DMDC  does  not  know  the  true  number  of  sexual 
assaults  in  the  US  military.  Many  sexual  assaults  are  not  reported  for  many  reasons  including 
potential  retaliation.  However,  reported  sexual  assaults  to  the  US  military  can  be  compared. 
DMDC  was  able  to  compare  the  number  of  sexual  assault  reports  filed  by  Reserve  and  National 
Guard  members  to  weighted  estimates  from  survey  respondents  to  assess  NRB  (and  overall  total 
survey  error). 

The  Sexual  Assault  Prevention  and  Response  Office  (SAPRO)  provided  DMDC  with 
summary  information  of  the  number  of  reported  sexual  assaults  (unrestricted  and  restricted)  to 
either  a  Service  Sexual  Assault  Response  Coordinator  (SARC)  or  the  National  Guard  SARC. 

The  report  containing  the  summary  information  was  collected  in  the  Defense  Sexual  Assault 
Incident  Database  (DSAID).  For  a  record  to  be  entered  into  DSAID,  the  survivor  needed  to 
complete  a  Victim  Reporting  Preference  Statement  (DDForm  2910)  that  indicates  whether  the 
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survivor  would  like  to  make  either  a  restricted  or  unrestricted  report.  The  information  is  then 
captured  in  DSAID.  DMDC’s  requested  the  number  of  sexual  assaults  reported  from  August  1, 
2014  through  October  31,  2015  (14  months  of  data).  DMDC  requested  14  months  of  data  in 
order  to  mirror  the  12  month  time  frame  corresponding  with  the  survey  administration  time  in 
the  following  three  time  periods: 

•  August  1,2014  through  July  31,2015 

•  September  1,2014  through  August  31,2015 

•  October  1,  2014  through  September  30,  2015 

The  2015  WGRR  survey  fielded  from  August  2015  through  October  2015  and  the 
summaries  shown  above  would  serve  as  a  comparable  12  month  period.  The  summary  files 
contained  information  about  the  number  of  sexual  assaults  by  Reserve  component  as  well  as  if 
the  report  type  was  restricted  or  unrestricted.  On  the  2015  WGRR  survey,  sexual  assault 
survivors  were  asked  follow-up  questions  to  determine  1)  if  they  filed  a  fonnal  report  of  sexual 
assault,  2)  type  of  report  filed,  and  3)  to  verily  the  sexual  assault  occurred  within  the  last  12 
months.  The  2015  WGRR  survey  questions  regarding  these  behaviors  are  displayed  in  Table  13. 
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Table  13. 

2015  WGRR  Reporting  Questions 


2015  WGRR  Reporting  Questions 

DoD  provides  two  types  of  sexual  assault  reports 

•  Restricted  reoorts  allow  Deooleto  aet  information,  collect 
evidence,  and  receive  medical  treatment  and 
counselina  without  startina  an  official  investiaation  of  the 
assault 

•  Unrestricted  reoorts  start  an  official  investiaation  in 
addition  to  allowing  the  services  available  in  restricted 
reporting 

179.  Did  you  officially  report  this  unwanted  event  to 
the  military?  This  could  have  been  either  a 
restricted  or  unrestricted  report. 

IE  Yes 

It  can  be  difficult  to  remember  the  exact  date  when  events 
occurred  In  this  study,  it  is  important  to  know  which  events 
happened  in  the  last  12  months,  and  which  events  happened 
earlier. 

201.  Thinking  about  when  the  event  occurred,  how 
certain  are  you  that  it  occurred  in  the  last  12 
months?  If  the  event  occurred  over  a  long  time, 
think  about  whether  it  ever  happened  after  [X 

Date] 

153  Definitely  occurred  AFTER  [X  Date] 

M  Not  sure  if  it  occurred  BEFORE  OR  AFTER  [X 

Date] 

E  No 

153  Definitely  occurred  BEFORE  [X  Date] 

DoD  provides  two  types  of  sexual  assault  reports 

•  Restricted  reoorts  allow  Deooleto  aet  information,  collect 
evidence,  and  receive  medical  treatment  and 
counselina  without  startina  an  official  investiaation  of  the 
assault 

•  Unrestricted  reoorts  start  an  official  investiaation  in 
addition  to  allowing  the  services  available  in  restricted 
reporting. 

180.  Did  you  make... 

Only  a  restricted  report? 

[X]  Only  an  unrestricted  report? 

A  restricted  report  that  turned  into  an 

I2SI  unrestricted  report? 

153  Or  were  you  not  sure  what  type  of  report  it  was? 

The  DSAID  summary  file  provided  by  SAPRO  contained  an  average  of  836  restricted 
and  unrestricted  reports  of  sexual  assault  within  the  Reserve  components  during  the  12  month 
periods.  Table  14  contains  a  summary  of  the  average  12  month  reports,  by  component  and  type 
of  report. 
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Table  14. 

Summary  of  Sexual  Assault  Reports  in  DSAID  by  Component 


Service/Component 

Restricted 

Unrestricted 

Total 

Army 

151 

451 

602 

National  Guard 

116 

315 

431 

Reserve 

35 

136 

171 

Navy 

14 

29 

43 

Reserve 

14 

29 

43 

Marine  Corps 

3 

19 

22 

Reserve 

3 

19 

22 

Air  Force 

59 

109 

168 

National  Guard 

37 

58 

95 

Reserve 

22 

51 

73 

Not  Available 

1 

0 

1 

National  Guard 

1 

0 

1 

Total 

228 

608 

836 

Note.  There  is  one  report  that  does  not  have  enough  information  to  determine  the  Service  and  is  categorized  in  the 
table  as  not  available 


DMDC  used  three  criteria  from  the  survey  to  compare  the  information  provided  by 
respondents  on  the  2015  WGRR  to  the  summary  of  actual  numbers  reported  in  a  12  month  period 
on  DSAID.  The  three  questions  were: 

•  Answered  “Yes”  to  reported  the  sexual  assault  (Q179), 

•  Answered  “restricted”  or  “unrestricted”  (Q180),  and 

•  Answered  confirms  the  sexual  assault  occurred  in  the  past  12  months  (Q201) 

There  were  169  respondents  from  the  survey  that  indicated  in  Q 179  they  had  reported  a 
sexual  assault.  Of  the  169  respondents,  1 19  indicated  the  sexual  assault  occurred  in  the  last  12 
months  (Q201)  and  that  they  filed  either  a  restricted  or  unrestricted  report.  The  weighted 
estimate  based  on  these  1 19  respondents  is  1,013,  compared  with  the  836  cases  from  DSAID. 
While  the  confidence  interval  from  the  survey  estimate  is  within  the  number  of  DSAID  cases,  the 
survey  has  overestimated  the  known  true  value  by  a  fairly  large  amount  (2 1 .2%).  The  survey 
estimates  for  restricted  and  unrestricted  reports  both  similarly  overestimate  the  true  values. 

There  were  40  responding  members  that  indicated  they  filed  a  restricted  report  and  79  members 
that  indicated  they  filed  an  unrestricted  report.  Table  15  shows  a  summary  of  the  number  of 
respondents,  estimates  from  the  survey,  lower  and  upper  95-percent  confidence  bounds,  and  the 
actual  number  of  reports  from  the  DSAID  database.  Although  the  true  number  of  reports  falls 
within  the  confidence  intervals  in  each  case,  the  weighted  estimate  is  consistently  higher. 
Potential  reasons  for  this  difference  could  be  that  responding  members  are  either  mistakenly 
saying  they  filed  a  report  (measurement  error)  or  it  is  possible  that  members  who  filed  a  report 
respond  at  higher  rates.  Another  potential  reason  for  overestimating  the  number  of  DSAID  cases 
was  the  survey  question  itself.  For  all  paper  surveys,  the  sexual  assault  measure  time  reference 
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was  a  static  date  (August  10,  2014).  This  is  different  from  the  web  form  that  used  a  dynamic 
date.  For  example,  someone  filling  out  the  paper  survey  could  have  responded  as  late  in  the 
fielding  period  as  October  9,  2015,  for  these  respondents  the  question  could  potentially  be 
relevant  for  more  than  the  intended  12  months.  In  all  situations  described,  the  actual  number 
falls  within  the  confidence  interval  and,  this  provides  some  evidence  that  estimates  have  less 
concern  for  NRB.  RAND  performed  a  similar  analysis  for  the  active  duty  survey  in  the  2014 
RMWS  and  found  evidence  that  the  true  number  reported  was  actually  more  than  the  estimate. 
This  led  RAND  to  conclude  that  sexual  assault  victims  that  report  are  less  likely  to  respond  to 
the  survey. 


Table  15. 

Estimated  vs.  Actual  Number  of  Reported  Sexual  Assaults 


Type  of  Sexual 
Assaults 

Number  of 
Respondents  that 
filed  a  Report? 

Weighted  Total 
Estimate  from 
Survey 

95%  Confidence 
Interval  Lower 
Bound  of 
Estimate 

95%  Confidence 
Interval  Upper 
Bound  of 
Estimate 

Number  of 
Reports  in 
DSAID 

Restricted 

40 

254 

176 

332 

228 

Unrestricted 

79 

759 

566 

953 

608 

Total 

119 

1,013 

805 

1,222 

836 

Note.  The  number  of  reports  in  DSAID  is  the  average  of  the  three  12-month  totals  within  DSAID  (August  1,  2014  through  July  3 1,  2015, 
September  1,  2014  through  August  31,  2015,  and  October  1,  2014  through  September  30,  2015). 


Likelihood  of  Victims  to  Respond  to  Surveys 

DMDC  detennined  NRB  could  exist  if  sexual  assault  survivors  that  made  either  an 
unrestricted  or  restricted  report  were  either  more  or  less  likely  to  complete  the  survey.  If  sexual 
assault  survivors  that  made  a  formal  report  were  more  likely  to  complete  the  survey  this  could 
provide  some  evidence  for  overestimating  questions  related  to  having  experienced  a  sexual 
assault.  While  DMDC  can  control  for  many  factors,  adjusting  experiencing  a  sexual  assault  in 
the  past  12  months  was  not  possible.  If  sexual  assault  survivors  that  filed  a  report  were  more  or 
less  likely  to  fill  out  the  survey,  this  could  be  evidence  for  over  or  under  estimating  questions 
related  to  sexual  assault.  This  analysis,  however,  is  limited  since  it  includes  only  those  sexual 
assault  survivors  that  made  an  unrestricted  report  of  an  assault.  If  those  sexual  assault  survivors 
that  do  not  report  an  assault  or  make  a  restricted  report  have  different  response  rate  behaviors 
than  those  that  make  an  unrestricted  report,  the  analysis  could  be  misleading.  However,  since 
DMDC  does  not  have  a  list  of  all  sexual  assault  survivors  this  subset  of  the  total  will  be  used. 

The  DSAID  file  provided  by  SAPRO  contained  1,044  records  of  which  752  were 
unrestricted  and  292  were  restricted.  The  restricted  reports  did  not  contain  names  or  social 
security  numbers  (SSN)  and  thus  the  potential  for  matching  was  the  752  records.  After  removing 
blank,  invalid,  and  duplicate  SSNs,  there  were  714  potential  records  to  match  to  the  sample. 
DMDC  matched  the  714  SSNs  to  the  2015  WGRR  sample  of  485,774  and  found  that  578 
matched.  DMDC  then  detennined  the  response  rate  for  these  matches  and  determined  a  sample 
weighted  response  rate  of  19.6  percent  was  associated  with  the  sexual  assault  survivors  that 
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reported  an  incident.  This  estimate  is  not  significantly  different  than  the  response  rate  of  19.9 
percent  overall.  Based  on  this  analysis,  there  is  no  evidence  to  conclude  that  sexual  assault 
survivors  who  made  an  unrestricted  report  are  more  or  less  likely  to  respond  to  the  survey. 


Table  16. 

Comparison  of  Response  Rates  from  Reported  Survivors  to  Full  Sample 


Full  Sample 

Sexual  Assault  Survivors  that  Reported 

Number  in  Sample 

Response  Rate 

Number  in  Sample 

Response  Rate 

485,774 

19.9% 

578 

19.6% 

Note.  The  estimates  are  based  on  the  752  unrestricted  reports  made 


Summary  of  Comparison  of  Known  Population  Values  with  Weighted  Survey 
Estimates 

The  purpose  of  this  section  of  the  NRB  analysis  is  to  detennine  whether  there  were  any 
differences  between  population  estimates  made  from  the  sample  respondents  compared  to  a 
source  with  trusted  data  (cases  in  the  DSAID  database).  Differences  between  population 
estimates  and  known  data  would  be  an  indication  of  possible  total  survey  error  caused  in  part  by 
nonresponse.  A  second  analysis  compared  response  rates  from  sexual  assault  survivors  who 
made  an  unrestricted  report  to  response  rates  from  all  members  of  the  survey.  Differences  in 
response  rates  would  indicate  sexual  assault  survivors  potentially  respond  at  different  rates  and 
since  they  cannot  be  controlled  in  the  weighting,  this  would  be  a  concern  for  NRB.  In  both 
cases,  there  were  no  reasons  to  draw  any  conclusions  that  NRB  existed  in  the  estimates  related  to 
sexual  assault.  The  overall  number  of  actual  sexual  assault  survivors  that  made  an  unrestricted 
or  restricted  report  fell  within  the  confidence  bound;  this  result  was  true  for  unrestricted  and 
restricted  reports  as  well.  In  addition,  the  response  rates  for  sexual  assault  survivors  that  made 
an  unrestricted  report  were  no  different  than  the  overall  sample.  From  this  analysis  DMDC 
concludes  there  is  little  or  no  evidence  of  NRB  associated  with  the  estimates  of  sexual  assaults. 


Section  2:  Analyze  Item  Missing  Data  and  Drop  Offs  for  Sexual  Assault 
Questions 

In  this  section,  DMDC  analyzed  item  missing  data  for  the  six  key  WGRR  1501  sexual 
assault  (SA)  behavior  questions  from  all  web  returns  to  investigate  whether  respondents  may 
refuse  these  sensitive  questions  at  different  rates.  In  addition,  DMDC  conducted  an  analysis  to 
investigate  if  certain  members  quit  the  survey  all  together  (i.e.,  drop-off)  because  of  the  sensitive 
and  graphic  sexual  assault  questions.  If  the  decision  to  refuse  to  answer  the  question  or  drop-off 
is  not  random  (i.e.,  those  who  avoid  the  SA  questions  have  different  sexual  assault  rates  than 
complete  respondents),  then  a  source  of  NRB  exists.  DMDC  cannot  directly  test  this  hypothesis 
because  the  sexual  assault  status  for  these  item  missing  cases  is  unknown.  However,  DMDC 
indirectly  assesses  NRB  by  examining  the  missing  data  patterns  and  characteristics  of  members 
who  drop-off  from  the  survey. 
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Item  Missing  Data  in  Sexual  Assault  Behavior  Questions 


The  six  SA  behavior  questions  are  displayed  in  Table  17. 


Table  17. 

2015  WGRR  Sexual  Assault  (SA)  Questions 


2015  WGRR  Sexual  Assault  Questions  (SA1  through  SA6)a 

67.  Since  IX  Datel,  did  vou  have  anv  unwanted 
experiences  in  which  someone  put  his  penis 
into  vour  lanus  or  mouthl  Tvaaina.  anus,  or 
mouth]? 

Yes 

3  No 

113.  Since  [X  Date],  did  vou  have  any  unwanted 
experiences  in  which  someone  intentionally 
touched  private  areas  of  vour  body  (either 
directly  or  through  clothing)?  Private  areas 
include  buttocks,  inner  thigh,  breasts,  groin,  anus, 
vagina,  penis,  or  testicles. 

3  Yes 

13  No 

81.  Since  TX  Datel,  did  vou  have  anv  unwanted 
experiences  in  which  someone  put  any  object 
or  anv  bodv  part  other  than  a  penis  into  vour 
[anus  or  mouth]  [vagina,  anus,  or  mouth]?  The 

body  part  could  include  a  finger,  tongue,  or 
testicles. 

3  Yes 

3  No 

129.  Since  TX  Datel,  did  vou  have  anv  unwanted 

experiences  in  which  someone  made  vou  touch 
private  areas  of  their  body  or  someone  else's 
body  (either  directly  or  through  clothing)?  This 
could  involve  the  person  putting  their  private  areas 
on  you  Private  areas  include  buttocks,  inner  thigh, 
breasts,  groin,  anus,  vagina,  penis,  or  testicles 

13  Yes 

3  No 

97.  Since  IX  Datel.  did  anvone  make  vou  put  anv 
part  of  your  body  or  any  object  into  someone's 
mouth,  vagina,  or  anus  when  you  did  not  want 

to?  A  part  of  the  body  could  include  your  [penis, 
testicles,  tongue,  or  fingers]  [tongue  or  fingers] 

3  Yes 

3  No 

145.  Since  [X  Datel,  did  vou  have  anv  unwanted 
experiences  in  which  someone  attempted  to 
put  a  penis,  an  object,  or  any  body  part  into 
your  [anus  or  mouth]  [vagina,  anus,  or  mouth], 
but  no  penetration  actually  occurred? 

3  Yes 

3  No 

a  The  2015  WGRR  incorporated  dynamic  text  in  the  SA  questions  to  reflect  “12  months  prior  dates”  based  on  when 
the  respondent  started  the  survey.  In  addition,  dynamic  text  was  also  used  based  on  the  gender  of  the  respondent. 
DMDC  uses  [  ]  to  indicate  in  questions  and  descriptions  where  dynamic  text  was  used. 


As  described  in  Table  3  (Case  Dispositions  for  Weighting),  485,774  sample  members 
were  selected  for  the  2015  WGRR.  Most  sampled  members  did  not  respond  to  the  survey 
(337,899  members,  69.5%);  this  is  typical  of  other  military  surveys  DMDC  has  conducted  and 
these  unit  nonrespondents  provide  no  information  for  this  analysis.  Unit  nonrespondents  include 
returning  a  blank  survey,  survey  refusals,  and  other  nonrespondents.  DMDC  keeps  data  on  the 
reasons  for  refusals  and  inspection  of  the  data  revealed  101  (-0.0%)  sample  members  identified 
the  refusal  reason  as  survey  content  was  intrusive.  Blank  surveys  (11  cases  by  web)  are  surveys 
that  are  returned  with  no  answered  questions;  respondent’s  motives  for  failing  to  start  the  survey 
(and  therefore  answers  to  the  SA  questions)  are  unknown,  but  DMDC  suspects  some  respondents 
have  learned  they  can  avoid  future  e-mail  follow-ups  by  submitting  a  blank  survey.  There  were 
46,592  (9.6%)  sampled  members  that  were  not  located  (i.e.,  e-mail  bounce  back  or  postal  non¬ 
deliverable).  In  addition,  12,171  (2.5%)  members  were  ineligible  to  complete  the  survey  (i.e.,  no 
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longer  a  member  of  the  Selected  Reserve  either  detennined  by  record  ineligibles  or  survey  or 
proxy  responses).  The  remaining  sample  count  of  complete  and  incomplete  (subsequently 
referred  to  as  partial)  survey  respondents  is  89,1 12  (18.3%). 

Table  18  shows  the  breakdown  of  missingness  for  the  complete  eligible  and  partial  web 
and  paper  respondents  for  the  six  SA  questions  for  web  and  paper  respondents.  The  Total 
Missing  column  indicates  the  number  of  missingness  for  each  of  the  questions  (ranging  from 
2.4%  to  3.3%).  The  following  columns  indicate  the  number  of  missing  complete  respondents 
(MC)  and  missing  partial  respondents  (MP).  There  are  the  same  number  of  MP  respondents 
(1,985)  for  each  SA  question  because  incomplete  eligibles  (SAMP_DC=5)  failed  to  answer  at 
least  one  sexual  assault  questions.  MC  respondents  answered  at  least  one  sexual  assault  question 
(at  least  one  question  from  SA1  through  SA6)  and  were  missing  on  one  or  more  other  sexual 
assault  questions.  In  addition,  this  information  is  broken  down  by  gender.  The  percentage  of 
missing  data  for  SA  questions  is  slightly  higher  for  females  (about  one  percentage  point  for  each 
SA  question).  Because  women  have  higher  sexual  assault  rates  than  men,  it  is  possible  that 
women  skip  the  SA  questions  at  higher  rates  because  a  sexual  assault  increased  the  likelihood 
that  they  avoid  these  questions.  If  this  were  true,  it  may  cause  some  underreporting  of  sexual 
assault  in  the  2015  WGRR.  It  is  important  to  note  that  the  missing  data  rates  for  the  SA  questions 
are  similar  to  other  base  questions  from  the  2015  WGRR  survey.  The  average  missing  data 
percentage  for  base  questions2  prior  to  the  sexual  assault  questions  is  1.5  percent.  The  average 
missing  data  for  base  questions  after  the  sexual  assault  question  is  4.4  percent. 

Questions  SA4  and  SA5  are  likely  less  sensitive  to  potential  respondents  than  the  other 
four  S  A  questions  because  they  avoid  the  graphic  description  of  body  parts  and  only  involve 
descriptions  of  touching.  Generally,  the  missing  data  rates  increase  in  a  relatively  linear  way  as 
respondents  progress  through  the  survey  due  to  survey  drop-offs.  Table  18  shows  that  the 
missing  completes  increase  from  SA1  through  SA3  but  decrease  at  SA4  and  SA5  (both  3.1%) 
before  increasing  again  to  SA6  (up  to  3.3%).  The  lower  missing  data  rates  for  the  less  sensitive 
questions  may  provide  some  evidence  that  the  more  graphic  SA  questions  caused  respondents  to 
either  skip  questions  or  quit  the  survey. 


2  Base  questions  were  defined  as  questions  that  were  not  embedded  in  any  skip  pattern 
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Table  18. 

Breakdown  of  Sample  Cases  to  Assess  Item  Missing  Data  for  Sexual  Assault  Questions 


Sexual 

Assault 

Question 

Answered 

Total 
Missing a 
(MC+MP) 

Missing 

Complete 

(MC) 

(SAMP_DC=4) 

Missing  Partial 
(MP) 

(SAMP_DC=5) 

Missing  By  Gender 

Yes 

No 

Male 

(n=53,421) 

Female 

(n=35,691) 

SA1 

(Q67) 

343 

86,593 

2,176 

(2.4%) 

191 

1,985 

1,102 

(2.1%) 

1,074 

(3.0%) 

SA2 

(Q81) 

235 

86,411 

2,466 

(2.8%) 

481 

1,985 

1,275 

(2.4%) 

1,191 

(3.3%) 

SA3 

(Q97) 

98 

86,122 

2,892 

(3.2%) 

907 

1,985 

1,531 

(2.9%) 

1,361 

(3.8%) 

SA4 

(QH3) 

1,033 

85,288 

2,791 

(3.1%) 

806 

1,985 

1,467 

(2.7%) 

1,324 

(3.7%) 

SA5 

(Q129) 

286 

86,038 

2,788 

(3.1%) 

803 

1,985 

1,489 

(2.8%) 

1,299 

(3.6%) 

SA6 

(Q145) 

198 

85,931 

2,983 

(3.3%) 

998 

1,985 

1,587 

(3.0%) 

1,396 

(3.9%) 

a  The  Total  Missing  denominator  is  89,1 12  members.  For  example,  in  the  SA1  row,  343  victims  (Yes)  +  86,593  No’s  +  2,176  Total  Missing 
equals  89,1 12. 


DMDC  explored  the  missing  data  patterns  for  “yes”  and  “no”  answers  to  each  SA 
question.  Table  19  shows  that  the  overwhelming  majority  of  respondents  who  answer  ‘Yes’  to 
an  SA  question  complete  all  other  SA  questions  (ranging  from  95.0%  to  98.0%).  This  analysis 
shows  that  SA  survivors  generally  answer  the  full  set  of  SA  questions.  Respondents  that  selected 
“No”  for  SA1-SA6  answer  the  full  sexual  assault  set  at  even  higher  rates  (ranging  from  97. 1%  to 
98.0%).  One  possibility  is  respondents  that  have  experienced  an  SA  behavior  may  be  less  likely 
to  answer  additional  sensitive  questions  because  it  may  provoke  negative  emotions.  A  second 
possibility  is  the  survey  is  simply  longer  for  the  SA  victims.  For  each  “yes”  to  SA1-SA6,  the 
respondent  is  presented  the  legal  criterial  item  bank  (11  to  13  item),  until  the  legal  requirement  is 
met.  For  example,  if  a  respondent  answered  “yes”  to  SA1,  they  see  1 1  legal  criteria  items.  If  the 
behavior  met  the  legal  criteria,  and  the  respondent  experienced  additional  behaviors  (SA2-SA6), 
they  are  not  presented  the  legal  criteria  item  bank  again.  Table  19  shows  limited  evidence  that 
some  SA  behaviors  may  be  slightly  underreported  because  SA  survivors  skip  SA  questions  at  a 
higher  rate  than  non-survivors. 
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Table  19. 

Missing  Data  Analysis  for  Answers  to  SA1-SA6 


Sexual 

Assault 

Question 

Answered  “Yes”  to  at  least  one  SA  Question 

Answered  “No”  to  at  least  one  SA  Question 

Answered 
“Yes”  to 
Question 

Completed  all 
other  SA 
questions 

Percent 
completed  all 
other  SA 
questions 

Answered 
“No”  to 
Question 

Completed  all 
other  SA 
questions 

Percent 
completed  all 
other  SA 
questions 

SA1  (Q67) 

343 

326 

95.0% 

86,593 

84,043 

97.1% 

SA2  (Q81) 

235 

228 

97.0% 

86,411 

84,141 

97.4% 

SA3  (Q97) 

98 

94 

95.9% 

86,112 

84,275 

97.9% 

SA4  (Q113) 

1,033 

993 

96.1% 

85,288 

83,376 

97.8% 

SA5  (Q129) 

286 

281 

98.3% 

86,038 

84,088 

97.7% 

SA6  (Q145) 

198 

194 

98.0% 

85,931 

84,175 

98.0% 

DMDC  examined  item  missing  data  from  the  2014  Status  of  Forces  Survey  of  Reserve 
Component  Members  ( 2014  SOFR)  to  compare  with  2015  WGRR  missing  data.  Both  surveys 
have  the  same  target  population  of  Reserve  and  National  Guard  members,  but  the  2014  SOFR 
questions  are  less  sensitive.  Table  20  shows  the  average  missing  rate  for  2014  SOFR  base 
questions  Q1-Q7 1  is  3.4  percent.  DMDC  used  base  questions  for  this  analysis  because  they  are 
seen  by  all  respondents  (i.e.,  unaffected  by  skip  patterns).  There  are  a  total  of  49  base  questions 
from  Q 1 -Q7 1 .  The  set  Q 1 -Q7 1  are  also  similar  in  location  within  the  questionnaire  to  the  1 8 
base  questions  within  the  2015  WGRR  (Q1-Q21).  The  average  missing  data  rate  for  the  2015 
WGRR  (Q1-Q21)  is  1.5  percent,  which  is  lower  than  the  2014  SOFR  base  questions. 

In  addition,  the  analysis  compares  key  questions  on  each  survey  with  single  item 
presentation  that  are  not  embedded  in  any  skip  logic.  DMDC  has  observed  from  prior  surveys 
across  different  questionnaire  content  that  group  items  and  questions  within  deep  skip  logic  have 
higher  rates  of  missing  data.  Therefore,  if  2015  WGRR  has  higher  missing  data  rates,  it  may  be 
due  to  question  sensitivity.  The  most  common  questions  of  interest  on  2014  SOFR  are  Q40, 
Q42,  and  Q55.  DMDC  finds  the  less  sensitive  2014  SOFR  questions  have  similar  missing  data 
rates  to  the  sensitive  2015  WGRR  questions.  For  example,  the  2014  SOFR  missing  data  rates  for 
questions  on  satisfaction,  retention,  and  stress  were  1.6,  1.7,  and  3.1  percent,  respectively.  The 
average  2015  WGRR  missing  rate  for  SA  questions  was  2.9  percent,  very  similar  to  the  2014 
SOFR  questions.  If  SA  questions  from  2015  WGRR  had  substantially  higher  missing  data  rates 
than  the  2014  SOFR,  it  may  provide  evidence  that  the  sensitive  questions  negatively  impact 
respondents,  and  perhaps  disproportionately  SA  victims.  However,  this  analysis  provides  no 
evidence  that  this  is  occurring. 
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Table  20. 

Comparison  Between  2015  WGRR  and  2014  SOFR  on  Missingness  for  Non-Sensitive  Items 


Survey 

Question 

Average 
Missing  Data 
Rate 

2014  SOFR 

Q1-Q71.  Base  items  -  Average  Missing  rate 

3.4% 

2015  WGRR 

Q1-Q21.  Base  items  -  Average  Missing  rate 

1.5% 

2014  SOFR 

Q40.  Overall,  how  satisfied  are  you  with  the  military  way  of 
life? 

1.6% 

2014  SOFR 

Q42.  Suppose  that  you  have  to  decide  whether  to  continue  to 
participate  in  the  National  Guard/Reserve.  Assuming  you  could 
stay,  how  likely  is  it  that  you  would  choose  to  do  so? 

1.7% 

2014  SOFR 

Q55.  Overall,  how  would  you  rate  the  current  level  of  stress  in 
your  military  life? 

3.1% 

2015  WGRR 

SA1-SA6  -  Average  Missing  rate 

2.9% 

Drop-off  Analysis  for  Sexual  Assault  Questions 

As  mentioned  previously,  partial  respondents  are  members  who  started  the  survey  but 
failed  to  answer  at  least  one  of  the  sexual  assault  questions.  DMDC  wanted  to  understand  why 
the  partial  respondents  did  not  complete  the  survey.  Specifically,  did  partial  respondents  avoid 
answering  any  of  the  SA  questions  due  to  their  sensitive  nature,  or  did  they  quit  the  survey  prior 
to  the  SA  questions  for  another  reason.  To  perform  this  analysis  DMDC  conducted  a  drop-off 
analysis. 

The  drop-off  analysis  shows  the  last  question  that  a  web  survey  respondent  answered  on 
the  survey  with  a  valid  response.  Drop-off  analysis  is  limited  strictly  to  web  respondents 
because  they  cannot  advance  to  see  further  questions  without  hitting  the  forward  button,  while 
paper  can  see  all  of  the  questions.  For  example,  if  a  respondent  answered  Ql-10  and  quit,  the 
drop-off  analysis  would  indicate  the  last  question  the  respondent  answered  was  Q10,  and  the  first 
question  they  saw  but  did  not  answer  was  Q1 1.  This  drop-off  analysis  does  not  account  for 
“standard  item  missing  data,”  for  instance  when  a  respondent  skips  one  question  (accidentally  or 
on  purpose),  but  returns  to  answer  further  questions.  For  instance,  if  a  member  answered  Ql-10, 
skipped  to  12  and  answered  Q 12-20,  and  then  answered  no  further  questions,  the  drop-off 
analysis  would  include  the  member  in  the  count  where  Q20  was  last  answered. 

Analysis  of  drop-offs  for  partial  respondents  is  only  considered  for  the  1,958  web 
respondents  because  it’s  impossible  to  determine  which  question  a  paper  respondent  was  viewing 
when  they  quit  the  survey.  Due  to  the  complexity  of  the  survey  instrument  and  the  defined 
sections  that  surround  certain  questions,  DMDC  grouped  survey  items  based  on  their  content  for 
questions  preceding  SA1  (Q67).  Four  content  modules  were  identified  by  DMDC  as  follows: 

1.  Demographics — Reserve  component  member  status  and  gender.  (Ql,  Q2) 
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2.  Time  Reference — Important  key  events  to  provide  frame  of  reference  for  respondents 
on  the  time  frame  of  “12  months  prior  to  taking  the  survey.”  (Q3-Q5) 

3.  Gender-Related  MEO  Violations — Experiences  of  MEO  violations  (gender 
discrimination  and  sexual  harassment)  in  the  12  months  prior  to  the  survey.  (Q6- 
Q21) 

4.  The  Gender-Related  MEO  Violation  with  Greatest  Effect — Circumstances  pertaining 
to  the  experience  of  MEO  violation(s)  in  the  past  12  months  that  had  the  greatest 
impact  on  the  respondent.  (Q22-Q66) 

Table  22  shows  the  drop-off  analysis  based  on  the  content  modules  defined  for  the  1,958 
partial  web  respondents.  It  is  important  to  note  that  drop-offs  are  in  the  minority  of  all 
respondents  who  started  the  survey.  For  the  modules  identified,  between  97.5  to  99.9  percent  of 
members  who  saw  those  specific  questions  answered  them.  In  addition,  these  rates  are  consistent 
across  other  DMDC  surveys  regardless  of  the  nature  of  the  content.  The  first  three  modules 
(Demographics,  Time  Reference  and  MEO  Questions)  are  all  base  items  and  respondents  see 
these  questions  in  a  linear  order.  As  respondents  progress  through  the  survey  the  level  of 
sensitive  infonnation  increases.  For  instance,  the  Demographics  module  asks  if  a  member  was 
part  of  the  Reserve  component  (Ql)  and  their  gender  (Q2).  This  infonnation  is  not  considered 
sensitive,  and  99.9  percent  of  all  of  those  who  saw  the  question  answered  it.  The  first  time 
reference  question,  “Do  you  currently  live  in  the  same  house  or  building  that  you  did  [a  year 
ago]?”  (Q3)  could  be  interpreted  as  a  sensitive  question  by  some,  and  makes  up  8.3  percent  of 
the  drop-offs. 

The  gender-related  MEO  violation  module  is  composed  of  sixteen  questions  asking  the 
member  about  gender-related  experiences  in  the  military.  These  questions  are  sensitive,  and 
include  a  question  that  could  potentially  be  classified  as  a  sexual  assault,  “Since  [a  year  ago],  did 
someone  from  work  intentionally  touch  you  in  a  sexual  way  when  you  did  not  want  them  to?” 
(Q16).  While  the  vast  majority  of  the  respondents  answered  these  questions  (98.5%  of  all 
respondents  who  saw  Q6  through  Q2 1  answered  them)  this  section  also  comprised  the  largest 
source  of  drop-offs  on  the  entire  survey.  Another  majority  of  drop-offs  exited  the  survey  in  the 
MEO  Violation  with  Greatest  Effect  module  of  questions  (27.6%  of  drop-offs).  The  drop-off 
pattern  was  relatively  consistent  attrition  throughout  the  MEO  section,  and  no  single  question 
had  an  extreme  spike  in  drop-offs. 

The  last  module  displayed  in  Table  22  is  the  first  sexual  assault  question,  SA1  (Q67). 
There  are  five  distinct  paths  respondents  can  take  to  view  the  first  sexual  assault  question  (SA1) 
based  on  skip  logic.  DMDC  determined  that  the  total  number  of  partial  respondents  who 
dropped  off  “most  likely”  while  viewing  the  first  sexual  assault  question  (SA1)  was  221 
members  (1 1.3  %  of  the  drop-offs).  This  question  was  determined  to  cause  the  single  largest 
number  of  drop-offs  on  the  survey.  However,  it  is  important  to  note  that  while  the  22 1  members 
are  the  largest  number  of  drop-offs  for  an  individual  question,  similar  numbers  of  members 
dropped  off  at  the  Time  Reference  section  (8.3%  of  the  drop-offs)  which  has  less  sensitive 
questions.  Interestingly,  Table  20  shows  that  males  tend  to  drop-off  at  much  higher  rates  during 
the  first  five  questions,  but  females  started  to  drop-off  at  higher  rates  through  the  MEO  section, 
but  this  effect  did  not  carry  through  to  the  SA1  question.  Because  the  SA1  question  creates  the 
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largest  number  of  drop-offs,  DMDC  interprets  this  as  being  of  some  concern  that  the  sensitive 
questions  have  higher  rates  of  missing  data;  however,  the  missing  data  rates  for  all  WGRR 
questions  are  still  low. 


Table  20. 

2015  WGRR  Drop-Off  Analysis 


Module 

Definition 

Number  of 
Drop-offs 

Number  of  Drop- 
offs  By  Gender 

Percent  of  all 
Drop-offs 
(n=l,958) 

Percent  of 
Total  who 
Answered  web 
Question  b 
(n=76,577) 

Male 

Female 

Demographics  (Q1-Q2) 

Last  question  was  Q 1 

65 

42 

23 

3.3% 

99.9% 

Time  Reference  (Q3-Q5) 

Last  question  was 
between  Q2  to  Q4 

163 

107 

56 

8.3% 

99.7% 

MEO  Questions  (Q6- 
Q21) 

Last  question  was 
between  Q5  to  Q20 

928 

519 

409 

47.4% 

98.5% 

MEO  Legal  Criteria 
(Q22-52),  MEO 

Violation  with  Greatest 
Effect  (Q54-Q66) 

Last  question  was 
between  Q21  to  Q64 
and  did  not  see  Q67 

540 

151 

389 

27.6% 

97.5% 

Sexual  Assault  (Q67) 

Last  question  was 
directly  before  Q67a 

221 

135 

86 

11.3% 

97.5% 

All  other  questions 

Last  question  was 
between  Q208  to 
Q236 

41 

25 

16 

1.9% 

91.5% 

Total 

1,958 

979 

979 

100% 

aRespondents  take  different  paths  to  the  Sexual  Assault  question  depending  on  the  answers  to  the  MEO  questions. 
Percent  who  completed  last  question  in  module 


Summary  of  Item  Missing  Data  and  Drop-off  for  Sexual  Assault 

DMDC  assessed  the  possible  effects  of  item  missing  data  on  NRB  through  an  analysis  of 
item  missing  data  across  the  six  SA  questions  on  the  2015  WGRR  as  well  as  conducting  an 
analysis  of  the  members  that  decided  to  drop-off  entirely  from  the  survey.  The  item  missing  data 
rates  on  the  2015  WGRR  are  comparable  to  SOFR  surveys,  and  therefore  it  does  not  appear  that 
the  graphic  and  sensitive  WGRR  questions  turn  off  large  numbers  of  respondents.  Most 
members  who  saw  the  sexual  assault  questions  answered  them  (between  97.5-99.5%),  but 
women  were  slightly  more  likely  than  men  to  skip  the  SA  questions  (~3%  compared  to  ~2%).  In 
addition,  the  pattern  of  drop-offs  throughout  the  survey  also  does  not  show  that  members  are 
offended  by  the  sensitivity  of  the  survey.  DMDC  assumes  that  some  NRB  is  introduced  because 
the  largest  single  drop-off  for  partial  respondents  was  directly  preceding  the  SA1  question,  and 
women  skipped  this  question  at  higher  rates  than  men.  However,  DMDC  also  believes  that  the 
impact  is  likely  small  as  other  non-sensitive  questions  have  similar  drop-off  rates. 
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Section  3:  Analysis  of  DMDC’s  Survey  of  Nonrespondents 

If  survey  respondents  and  nonrespondents  have  different  sexual  assault  propensities  that 
cannot  be  accounted  for  during  survey  weighting,  it  would  result  in  biased  survey  estimates  of 
sexual  assault.  DMDC  conducted  a  nonresponse  study  ( 2015  WGRR-N)  based  on  a  sample  of 
nonrespondents  from  the  original  survey  as  another  method  to  assess  NRB.  The  2015  WGRR-N 
was  a  web  questionnaire  with  only  e-mail  notifications  (the  2015  WGRR  had  postal  notifications 
and  paper  form),  and  the  questionnaire  was  much  shorter  than  the  2015  WGRR.  One  purpose  of 
the  survey  was  to  evaluate  NRB  by  comparing  the  responses  from  the  follow-up  survey  to  the 
original  survey.  In  particular,  there  were  four  questions  that  were  asked  on  both  the  2015  WGRR 
and  2015  WGRR-N  that  could  be  used  to  assess  NRB  (see  Table  21). 


Table  21. 

2015  WGRR-N  Comparison  Questions 


2015  WGRR-N  Comparison  Questions 

211.  To  what  extent  do/would  you  feel  safe  from 
being  sexually  assaulted  at  your  home  duty 
station? 

X  Very  safe 

X  Safe 

X  Neither  safe  nor  unsafe 

X  Unsafe 
|X1  Very  unsafe 

223.  Suppose  that  you  have  to  decide  whether  to 
continue  to  participate  in  the  National 
Guard/Reserve.  Assuming  you  could  stay,  how 
likely  is  it  you  would  choose  to  do  so? 

X  Very  likely 

X  Like|y 

X  Neither  likely  nor  unlikely 

X  Unlikely 

X  Very  unlikely 

212.  To  what  extent  do/would  you  feel  safe  from 
being  sexually  assaulted  during  military 
operations,  training,  or  exercises  away  from 
your  home  duty  station? 

IX]  Very  safe 

X  Safe 

X]  Neither  safe  nor  unsafe 

IX  Unsafe 

X  Very  unsafe 

224.  In  general,  would  you  say  your  health  is... 

X  Excellent? 

X  Very  good? 

X  Good? 

X Fair? 

X  Poor? 

If  estimates  from  these  matching  questions  were  significantly  different,  this  could  be 
evidence  of  NRB  in  these  questions  (and  potentially  other  correlated  questions).  As  described 
earlier  in  the  report,  a  sample  of  485,774  was  selected  from  the  population  of  817,007.  There 
were  336,1 12  nonrespondents  (SAMP_DC=1 1)  from  the  2015  WGRR  and  a  sample  of  59,973 
was  selected  for  the  2015  WGRR-N  to  assess  both  objectives  described  earlier.  The  sample  was 
selected  using  the  same  sampling  strata  as  the  2015  WGRR ,  since  there  were  enough 
nonrespondents  per  stratum,  and  the  sampling  tool  was  used  to  determine  the  sample  allocation. 
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DMDC  created  sampling  disposition  codes  using  the  criteria  shown  in  Table  22.  The  table 
shows  that  1,330  of  the  sample  were  considered  to  be  complete  eligible  based  on  the  criteria. 


Table  22. 

Sample  Disposition  Codes  for  2015  WGRR-N 


Sample  Disposition  Code 

(SAMPDC) 

Sample  Disposition  Code  Description 

Number  of  Members 

2 

Self  Report  Ineligibles:  If  the  respondent 
answered  “No”  to  Question  1  on  the  survey 
“Were  you  a  member  of  a  Reserve  component 
on  November  30,  2015?” 

25 

4 

Complete  Eligibles:  If  respondent  was  eligible 
and  completed  50%  or  more  of  the  questions. 

1,330 

5 

Incomplete  Eligibles:  If  respondent  was  eligible 
but  failed  to  complete  50%  or  more  of  the 
questions 

97 

11 

Nonrespondents:  All  others 

58,521 

Total 

59,973 

Table  23  shows  population,  sample  size,  respondents,  and  response  rate  by  key  domains 
(e.g.,  gender)  for  both  the  2015  WGRR  and  2015  WGRR-N.  The  weighted  response  rates  for  both 
surveys  have  similar  patterns  (although  rates  are  much  lower  for  the  2015  WGRR-N).  For 
example,  Air  National  Guard  and  Air  Force  Reserve  were  the  highest  responders  for  both  the 
2015  WGRR  (31.1%  and  27.4%)  and  2015  WGRR-N  (6.4%  and  4.6%).  Warrant  Officers 
responded  at  the  highest  rate  for  both  surveys,  although  for  2015  WGRR-N  the  number  of 
respondents  is  relatively  small  (less  than  30). 
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Table  23. 

Comparison  of  2015  WGRR  Sample  with  Nonresponse  Sample  2015  WGRR-N 


Domain 

Variable 

Domain 

Population 

2015  WGRR 

WGRR-N 

Sample 

Size 

Eligible 

Responses 

Weighted 

Response 

Rates 

Sample 

Size 

Eligible 

Responses 

Weighted 

Response 

Rates 

Sample 

Full  Sample 

817,007 

485,774 

87,127 

19.8% 

59,973 

1,330 

2.2% 

Reserve 

Component 

(RORGCD) 

Army  National 
Guard 

348,599 

186,481 

25,172 

16.1% 

13,608 

153 

1.1% 

US  Army 

Reserve 

197,698 

121,036 

18,674 

17.2% 

11,746 

206 

1.8% 

58,227 

36,245 

8,053 

24.1% 

8,016 

218 

2.7% 

US  Marine  Corps 
Reserve 

38,468 

36,364 

4,002 

12.2% 

13,852 

56 

0.4% 

Air  National 
Guard 

104,818 

61,695 

19,195 

31.8% 

6,085 

392 

6.4% 

US  Air  Force 
Reserve 

69,197 

43,953 

12,031 

28.2% 

6,666 

305 

4.6% 

Gender 

(RSEX2) 

Male 

662,565 

331,332 

52,421 

19.0% 

36,525 

654 

2.9% 

Female 

154,442 

154,442 

34,706 

23.3% 

23,448 

676 

2.9% 

Paygrade 

Grouping 

(RPAYGRP9) 

E1-E4 

352,772 

238,102 

18,575 

7.8% 

38,215 

380 

1.0% 

E5-E9 

336,347 

179,140 

45,178 

26.5% 

16,074 

644 

4.0% 

W1-W5 

12,187 

5,773 

2,203 

39.4% 

269 

18 

6.7% 

01-03 

59,530 

33,684 

9,125 

27.0% 

3,005 

135 

4.5% 

04-06 

56,171 

29,075 

12,046 

43.3% 

2,410 

153 

6.3% 

Reserve 

Program 

(RPROG1) 

TPU 

666,695 

414,431 

57,902 

14.7% 

54,245 

966 

1.8% 

AGR 

76,747 

33,432 

13,773 

42.7% 

2,692 

164 

6.1% 

MilTech 

61,484 

29,273 

12,737 

43.4% 

1,737 

151 

8.7% 

IMA 

12,082 

8,638 

2,715 

33.3% 

1,299 

49 

3.8% 

Weighting  the  2015  WGRR-N 

Because  some  members  were  not  eligible  for  the  2015  WGRR-N  sample,  it  is  not  a 
probability  sample  of  all  Reserve  members  and  there  is  no  method  to  create  base  weights. 
However,  weights  can  be  constructed  that  approximately  represent  the  Reserve  component 
population  by  direct  post-stratification.  DMDC  weighted  the  2015  WGRR-N  using  the  following 
process: 

•  Complete  eligibles  were  placed  into  post-stratification  cells  based  on  a  cross 
classification  of  gender  (male/female),  paygrade  (E1-E4,  E5-E9,  Wl-03,  04-06), 
and  component  (Army  Reserve,  Army  National  Guard,  Navy  Reserve,  Marine  Corps 
Reserve,  Air  Force  Reserve,  and  Air  National  Guard). 

•  Population  totals  for  these  initial  post-stratification  cells  were  determined  based  on 
the  total  sampling  population  for  the  2015  WGRR  survey. 
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•  Post-stratification  cells  were  combined  if  any  had  less  than  20  complete  eligibles  and 
were  collapsed  only  across  Reserve  component  levels.  The  rationale  for  combining  at 
the  component  level  was  that  experiences  of  males  who  are  E1-E4  are  more  likely  to 
be  more  similar  across  component  than  any  other  potential  collapsing.  One  example 
was  that  male/El-E4  for  Army  Reserve,  Army  National  Guard,  and  Marine  Corps 
Reserve  were  combined  for  44  complete  eligible  respondents.  After  collapsing, 
DMDC  created  26  post-stratification  cells3. 

•  DMDC  computed  final  weights  by  dividing  the  total  population  size  by  the  number  of 
complete  eligibles  within  the  collapsed  strata  (Nh/nh). 

Table  24  shows  the  estimates  and  corresponding  margins  of  error  for  the  four  questions 
that  overlapped  between  the  two  surveys.  For  example,  for  Retention,  DMDC  estimated  that 
78.0  percent  of  Selected  Reserve  members  indicated  they  were  likely  or  very  likely  to  stay  in  the 
National  Guard/Reserve  from  the  2015  WGRR  and  77.7  percent  indicated  this  on  the  2015 
WGRR-N.  The  margins  of  error  for  the  2015  WGRR  are  small,  whereas  they  are  larger  for  the 
WGRR-N  because  of  small  number  of  respondents.  Estimates  for  all  questions  are  extremely 
close  except  the  Health  question,  and  the  confidence  intervals  overlap  even  for  the  Health 
question. 


Table  24. 

Comparison  of  WGRR  Survey  with  Nonresponse  Study  Control  Questions 


Question 

Weighted  Estimates 

WGRR  Survey 
(n=87,127) 

Nonresponse  Survey 
(n=l,330) 

RETENTION:  Assuming  you  could  stay  [in  the 
National  Guard/Reserve],  how  likely  is  it  you  would 
choose  to  do  so?  (%  Saying  Likely  or  Very  Likely) 

78.0%±1.0 

77.7%±5.1 

HEALTH:  In  general,  would  you  say  your  health  is. . .? 
(%  Saying  Good  or  Excellent) 

77.0%±1.0 

72.3%±5.2 

SAFETY  AT  HOME:  To  what  extent  do/would  you 
feel  safe  from  being  sexually  assaulted  at  your  home 
duty  station?  (%  Saying  Safe  or  Very  Safe)  -  note 
flipped  from  TAB  SHELL 

96.0%±1.0 

96.7%±1.5 

SAFETY  AWAY:  To  what  extent  do/would  you  feel 
safe  from  being  sexually  assaulted  during  military 
operations,  training,  or  exercises  away  from  -  note 
flipped  from  TAB  SHELL  your  home  duty  station? 

94.0%±1.0 

93.7%±2.4 

The  similar  estimates  from  the  2015  WGRR  and  nonresponse  study  fail  to  detect  any 
evidence  of  NRB  in  production  estimates  for  these  four  questions.  DMDC  further  researched 
whether  any  inferences  could  be  made  about  the  sexual  assault  questions.  Two  of  the  four 
questions  relate  to  safety  at  home  and  away  related  to  sexual  assault,  so  DMDC  hypothesized  a 


3  Details  of  the  weighting  and  collapsing  of  post-strata  are  available  upon  request. 
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relationship  with  the  sexual  assault  questions.  To  investigate  this,  DMDC  examined  correlations 
between  the  sexual  assault  questions  and  these  four  questions  from  the  2015  WGRR.  DMDC 
developed  a  sexual  assault  measure  (SA_R_ADJ)  based  on  answers  to  the  six  sexual  assault 
questions  and  their  corresponding  legal  definitions.  There  were  965  sexual  assault  survivors 
identified  from  the  survey  and  648  answered  the  safety  at  home  question.  Using  the  Phi- 
Coefficient  (binary  correlation)  the  unweighted  estimated  correlation  was  -0.1568  and  the 
weighted  correlation  was  -0.1791  (females  unweighted  was  -0.15  and  males  was  -0.14).  The 
negative  correlation  was  expected  because  SA  victims  should  feel  less  safe.  For  safety  away,  the 
correlations  were  both  negatively  correlated  and  nearly  identical  (-0.14  and  -0.17  unweighted 
and  weighted). 

Summary  of  Analysis  of  DMDC’s  survey  of  nonrespondents 

The  purpose  of  this  NRB  analysis  was  to  compare  estimates  from  four  identical  questions 
asked  on  the  2015  WGRR-N  and  2015  WGRR  to  assess  NRB.  If  estimates  were  substantively 
and  statistically  different,  this  would  be  evidence  of  NRB  for  these  estimates  in  the  2015  WGRR. 
Given  the  small  number  of  respondents  and  larger  margins  of  error  (MOEs)  in  the  2015  WGRR- 
N,  the  four  estimates  are  very  similar  across  the  two  studies.  Estimates  from  the  two  surveys  are 
within  1  percentage  point  of  each  other  for  three  of  the  four  questions  (Health  has  a  4.7 
percentage  point  difference)  but  the  MOE  is  large  for  2015  WGRR-N,  and  all  estimates  have 
overlapping  confidence  intervals.  This  study  fails  to  detect  any  evidence  of  NRB  in  production 
estimates  for  these  four  questions.  Additionally,  there  is  some  indication  that  measures 
correlated  with  these  questions  may  also  have  low  levels  of  NRB.  While  the  SA  questions  are 
correlated  with  two  measures  from  the  WGRR-N,  the  correlations  are  fairly  small  and  DMDC 
advises  against  drawing  conclusions  for  the  SA  questions. 


Section  4:  Evaluate  the  Sensitivity  of  Different  Post-Survey  Adjustments 
(Weighting  Methods)  on  Survey  Estimates 

Production  weights  for  the  2015  WGRR  were  produced  by  Westat  by  first  developing 
models  that  account  for  each  member’s  propensity  of  experiencing  unwanted  sexual  behaviors, 
and  then  using  those  estimated  propensities  throughout  the  weighting  process.  This  method  is 
consistent  with  RAND’s  approach  for  the  2014  RMWS,  but  represents  a  change  from  how 
DMDC  previously  conducts  survey  weighting.  DMDC  independently  developed  a  set  of  weights 
using  our  typical  methods  to  assess  the  effects  of  different  weighting  approaches  on  survey 
estimates.  This  section  uses  the  DMDC  weights  as  a  validity  check  to  determine  if  large 
differences  in  the  weights  exist,  and  if  these  potential  differences  lead  to  more  or  less  NRB  in 
survey  estimates. 

DMDC  Weighting  Methodology 

DMDC’s  standard  weighting  procedures  have  many  similarities  to  the  methods  recently 
used  by  RAND  and  Westat.  Both  methods  estimate  response  propensities  and  make  weighting 
adjustments  based  on  the  inverse  of  those  propensities.  However,  there  are  two  key  differences; 
first,  RAND  and  Westat  used  machine  learning  programs  called  Generalized  Boosted  Models 
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(GBM)  to  estimate  the  propensities  while  DMDC  uses  logistic  models  with  single  classification 
trees  (CHAID).  Second,  RAND  and  Westat  first  estimate  propensities  for  several  sexual  assault 
characteristics,  and  then  use  those  estimated  propensities  to  predict  survey  response.  DMDC 
skips  this  step  and  directly  models  survey  eligibility  and  response  propensities.  DMDC  weighted 
the  2015  WGRR  using  three  main  steps: 

•  Step  1 :  Adjust  weights  for  nonresponse  based  on  eligibility  as  follows: 

—  Transfer  the  weight  of  the  384,491  nonrespondents  (SAMP_DC  =  8,  9,  10,  1 1)  to 
the  88,668  cases  with  known  eligibility  (SAMP_DC  =  2,  3,  4,  5).  CHAID  (Chi- 
squared  Automatic  Interaction  Detection),  a  decision  tree  technique  based  on  Chi- 
square  tests,  was  used  to  determine  the  best  predictors  for  the  logistic  model.  A 
logistic  regression  model  was  used  to  predict  the  probability  of  eligibility  for  the 
survey  (known  eligibility  vs.  unknown  eligibility).  Weighting  adjustment  factors 
for  eligibility  were  computed  as  the  inverse  of  the  logistic  model-predicted 
probabilities.  The  model  was  weighted  using  the  sampling  weight  (base  weight). 
Predictors  in  the  eligibility  model  were  the  same  variables  used  by  Westat  with 
the  exception  of  the  model-predicted  probabilities  for  unwanted  sexual  behaviors. 

•  Step  2:  Adjust  weights  for  survey  completion  as  follows: 

—  Transfer  the  eligibility  weight  (created  in  Step  1)  of  the  1,985  incomplete  survey 
responses  (SAMP_DC  =  5)  to  the  87,127  complete-eligible  respondents 
(SAMP_DC  =  4).  Weighting  adjustments  for  completion  use  the  same 
methodology  as  Step  1  (CHAID  and  logistic  model). 

•  Step  3:  Create  final  weights 

-  The  weights  were  poststratified  to  match  population  totals  and  to  reduce  variance 
and  bias  unaccounted  for  by  the  previous  weighting  adjustments.  DMDC 
calculated  the  final  weight  as  the  product  of  adjustment  factors  in  Steps  1,  2  and 
3.  Poststratification  cells  were  defined  by  the  cross-classification  of  Service, 
gender,  paygrade  and  reserve  program.  Many  of  the  crossings  were  collapsed 
since  the  goal  was  to  create  poststratification  cells  with  more  than  30  respondents. 
Within  each  post-stratification  cell,  the  non-response-adjusted  weights  for  eligible 
respondents  and  self-reported  ineligibles  (SAMP_DC  =  2,  3,  4)  were  adjusted  to 
match  population  counts.  There  were  121  poststratification  cells. 

Comparison  of  Adjustment  Stages  and  Final  Weights 

Table  25  compares  the  DMDC  and  Westat  methods  for  each  of  the  weight  adjustments 
discussed  in  Steps  1  through  3:  eligibility,  completion,  and  poststratification.  The  comparison 
shows  the  univariate  distribution  of  each  weighting  adjustment  factor.  The  results  indicate  that 
some  edge  cases,  such  as  the  maximum  or  minimum  adjustments,  do  differ  in  both  the  eligibility 
model  and  poststratification  adjustments.  The  maximum  cell  in  the  DMDC  method  had  a  much 
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larger  eligibility  adjustment  than  the  Westat  method  (239.00  to  45.4 1)4.  In  addition,  the  DMDC 
method  also  differs  in  the  minimum  poststratification  adjustment  (0.47  to  0.77).  However,  while 
these  differences  do  exist  for  extreme  cases,  the  table  in  aggregate  is  very  similar  even  when 
considering  the  99  percent  and  one  percent  quantile  values  in  the  univariate  distribution. 
Furthermore,  the  mean  value  for  each  adjustment  is  nearly  identical.  Although  the  DMDC 
method  carries  slightly  more  variance,  all  adjustments  outside  of  the  tails  seem  to  convey  the 
same  adjustments  in  both  instances. 


Table  25. 

Comparison  between  DMDC  and  Westat  Weighting  Methods  for  Eligibility,  Completion  and 
Poststratification  Adjustments 


DMDC  Method  (Adjustment  Factors) 

Westat  Method  (Adjustment  Factors) 

Statistic 

Eligibility 

Completion 

Poststratification 

Eligibility 

Completion 

Poststratification 

Mean 

5.15 

1.02 

1.01 

5.14 

1.02 

1.03 

Standard 

Deviation 

6.18 

0.01 

0.05 

4.69 

0.01 

0.08 

100%  Max 

239.00 

1.19 

1.37 

45.41 

1.18 

1.36 

99% 

27.74 

1.07 

1.12 

22.00 

1.05 

1.29 

95% 

17.44 

1.04 

1.08 

16.00 

1.04 

1.14 

90% 

10.87 

1.04 

1.07 

11.80 

1.03 

1.12 

75%  Q3 

5.24 

1.03 

1.05 

6.03 

1.03 

1.09 

50%  Median 

2.98 

1.02 

1.02 

3.13 

1.02 

1.02 

25%  Q1 

2.12 

1.01 

0.98 

2.21 

1.02 

0.96 

10% 

1.72 

1.01 

0.96 

1.82 

1.02 

0.91 

5% 

1.55 

1.01 

0.94 

1.67 

1.02 

0.89 

1% 

1.44 

1.00 

0.86 

1.46 

1.00 

0.88 

0%  Min 

1.20 

1.00 

0.47 

1.09 

1.00 

0.77 

Table  26  extends  the  comparison  of  DMDC  and  Westat  methods  by  showing  the 
distribution  of  final  weights.  The  final  weight  takes  into  account  all  of  the  previous  weighting 
adjustments.  Again,  DMDC  sees  somewhat  erratic  behavior  at  the  tails  for  the  maximum  weight 
value,  but  are  very  close  in  most  of  the  other  quantiles.  DMDC  concludes  that  overall  both 
methods  produce  similar  distributions  of  weights. 


4  DMDC  completed  this  set  of  weights  for  the  purposes  of  comparing  and  contrasting  with  the  Westat  method.  In  a 
production  method,  DMDC  would  have  adjusted  the  larger  eligibility  adjustment  to  be  closer  to  Westat’s  value.  The 
large  cell  was  caused  by  an  extremely  large  number  of  nonrespondents  detected  in  a  particular  combination  of  input 
variables,  and  the  large  weight  was  only  in  that  specific  cell  (as  indicated  by  the  99%  quantile). 
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Table  26. 

Comparison  between  DMDC  Method  Final  Weights  and  Westat  Method  Final  Weights 


DMDC  Method 

Westat  Method 

Mean 

9.1 

9.1 

Standard  Deviation 

11.3 

OO 

oo 

100%  Max 

483.9 

86.2 

99% 

51.1 

43.8 

95% 

30.8 

29.0 

90% 

18.7 

19.2 

75%  Q3 

10.1 

11.5 

50%  Median 

5.7 

6.1 

25%  Q1 

3.2 

3.4 

10% 

2.0 

2.1 

5% 

1.7 

1.8 

1% 

1.4 

1.5 

0%  Min 

1.1 

1.1 

Comparison  of  Key  Estimates 

Finally,  DMDC  compared  differences  in  weighted  survey  estimates  based  on  the  DMDC 
and  Westat  sets  of  weights.  The  final  comparison  between  weighting  method  is  based  on  the 
key  survey  estimates  regarding  sexual  assault  questions.  Table  27  and  Table  28  shows  seven 
estimates  associated  with  sexual  assault  and  sexual  harassment  for  females  and  males.  For 
example,  the  estimates  of  any  sexual  assault  occurring  with  females  using  DMDC  standard 
methods  was  3.5  percent  and  the  estimate  was  3.4  percent  using  the  GBM  weighting  approach. 
All  comparisons  are  nearly  identical  for  both  weighting  approaches  and  the  largest  difference  in 
the  female  estimate  table  is  the  Gender  Discrimination  results  which  differ  by  0.2  percentage 
points. 
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Table  27. 

Comparison  of  DMDC  and  Westat  Key  Survey  Estimates  (Female  Only) 


Question 

Variable 

DMDC  Estimate 

Westat  Estimate 

Sexual  Quid  Pro  Quo 

QPQ 

1.4%  ±0.1% 

1.4%  ±0.1% 

Sexual  Assault-Penetrative 

SA_PEN 

1.4%  ±0.1% 

1.4%  ±0.1% 

Sexual  Assault-Any  Type 

SARATE 

3.5%  ±  0.2% 

3.4%  ±0.1% 

Sexual  Assault- Attempted  Touch 

SATOUCH 

1.9%  ±0.1% 

1.8%  ±0.1% 

Gender  Discrimination 

SDISC 

11.1%  ±0.3% 

10.9%  ±  0.2% 

Sexual  Harassment 

SEXHAR 

18.5%  ±0.4% 

18.6%  ±0.3% 

Sexual  Assault  Rate-Adjusted  for 
telescoping 

SARADJ 

3.2%  ±  0.2% 

3.2%  ±0.1% 

Similarly  the  largest  difference  in  the  male  estimates  is  the  Sexual  Harassment  question, 
which  also  differs  by  0.2  percentage  points.  Overall  all  of  the  other  estimates  are  nearly 
identical.  In  addition,  all  of  the  confidence  intervals  for  either  weighting  method  overlap. 


Table  28. 

Comparison  of  DMDC  and  Westat  Key  Survey  Estimates  (Male  Only) 


Question 

Variable 

DMDC  Estimate 

Westat  Estimate 

Sexual  Quid  Pro  Quo 

QPQ 

0.2%  ±  0.03% 

0.2%  ±  0.03% 

Sexual  Assault-Penetrative 

SAPEN 

0.2%  ±  0.03% 

0.2%  ±  0.03% 

Sexual  Assault-Any  Type 

SARATE 

0.6%  ±  0.05% 

0.7%  ±  0.06% 

Sexual  Assault- Attempted  Touch 

SATOUCH 

0.4%  ±  0.04% 

0.4%  ±  0.04% 

Gender  Discrimination 

SDISC 

1.5%  ±0.07% 

1.6%  ±0.07% 

Sexual  Harassment 

SEXHAR 

4.2%  ±0.15% 

4.4%  ±0.14% 

Sexual  Assault  Rate-Adjusted  for 
telescoping 

SARADJ 

0.5%  ±  0.05% 

0.5%  ±  0.05% 

Summary  of  Evaluate  the  Sensitivity  of  Different  Post-Survey  Adjustments 
(Weighting  Methods)  on  Survey  Estimates 

The  DMDC  and  Westat  weighting  methods  were  conducted  independently  using  different 
software  (DMDC  used  SAS  and  SPSS;  Westat  used  R  and  SAS)  and  methodology,  but  the 
overall  results  are  strikingly  similar  across  all  of  the  intermediate  weighting  steps,  the  final 
weights,  and  a  comparison  of  key  estimates.  In  addition,  all  of  the  confidence  intervals  for  either 
weighting  method  overlap.  In  conclusion  we  see  very  little  potential  for  bias  in  the  final  weights 
and  resulting  estimates  that  these  weights  were  used  to  produce. 
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Appendix  A. 

Domain  Based  Sampling  Size  and  Expected 

Response 


Domain 

Population 

Approximate 
Sample  Size1 

Expected 

Responses 

Percent 

Sampled 

All  Domains 

817,007 

486,119 

131,959 

59.5 

National  Guard 

453,417 

248,019 

66,370 

54.7 

Army  National  Guard 

348,599 

186,500 

41,676 

53.5 

Air  National  Guard 

104,818 

61,738 

24,694 

58.9 

Reserve 

363,590 

237,424 

65,589 

65.3 

US  Army  Reserve 

197,698 

120,991 

32,651 

61.2 

US  Navy  Reserve 

58,227 

36,217 

12,492 

62.2 

US  Marine  Corps  Reserve 

38,468 

36,352 

3,710 

94.5 

US  Air  Force  Reserve 

69,197 

43,940 

16,736 

63.5 

Enlisted 

689,119 

416,917 

101,433 

60.5 

E1-E4 

352,772 

238,121 

36,049 

67.5 

E1-E3 

158,071 

110,650 

16,211 

70.0 

E4 

194,701 

127,529 

19,838 

65.5 

E5-E9 

336,347 

179,273 

65,384 

53.3 

E5-E7 

307,320 

164,724 

59,639 

53.6 

E8-E9 

29,027 

14,339 

5,745 

49.4 

Officers 

127,888 

68,548 

30,526 

53.6 

01-03 

59,524 

33,691 

12,908 

56.6 

04-06 

56,171 

29,097 

15,077 

51.8 

W1-W5 

12,193 

5,779 

2,541 

47.4 

TPU 

666,703 

414,689 

96,828 

62.2 

AGR 

76,746 

33,461 

16,606 

43.6 

IMA 

12,048 

8,614 

3,709 

71.5 

Deployed  Last  12  Months 

33,507 

18,697 

5,596 

55.8 

Not  Deployed  Last  12  Months 

783,500 

466,966 

126,363 

59.6 

Non-Hispanic  White 

549,466 

313,196 

85,082 

57.0 

Total  Minority 

267,541 

172,564 

46,877 

64.5 

Non-Hispanic  Black 

129,665 

85,838 

24,939 

66.2 

Hispanic 

88,750 

55,824 

13,347 

62.9 

Females 

154,442 

154,442 

71,370 

100.0 

Females*Enlisted 

130,132 

130,132 

56,289 

100.0 

Females*El-E4 

71,779 

71,779 

22,670 

100.0 

Females*E5-E9 

58,353 

58,353 

33,619 

100.0 

Females*Officers 

24,310 

24,310 

15,081 

100.0 

Females*01-03 

13,165 

13,165 

7,164 

100.0 

Females*04-06 

9,784 

9,784 

6,983 

100.0 

Females*TPU 

125,127 

125,127 

52,575 

100.0 

Females*AGR 

15,725 

15,725 

9,868 

100.0 

Females*IMA 

3,145 

3,145 

1,906 

100.0 

Females*Non-Hispanic  White 

84,666 

84,666 

40,699 

100.0 

Females*  Minority 

69,776 

69,776 

30,671 

100.0 

Females*National  Guard 

76,588 

76,588 

34,282 

100.0 

Females*Army  National  Guard 

55,983 

55,983 

22,194 

100.0 

Females*Army  National  Guard*Enlisted 

50,044 

50,044 

18,680 

100.0 
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Domain 

Population 

Approximate 
Sample  Size1 

Expected 

Responses 

Percent 

Sampled 

Females*Army  National  Guard*Officers 

5,939 

5,939 

3,514 

100.0 

Females*Air  National  Guard 

20,605 

20,605 

12,088 

100.0 

Females*Air  National  Guard*Enlisted 

17,784 

17,784 

10,336 

100.0 

Females*Air  National  Guard*Officers 

2,821 

2,821 

1,752 

100.0 

Females*Reserve 

77,854 

77,854 

37,088 

100.0 

Females*US  Army  Reserve 

45,138 

45,138 

19,938 

100.0 

Females*US  Army  Reserve*Enlisted 

36,365 

36,365 

14,561 

100.0 

Females*US  Army  Reserve*Officers 

8,773 

8,773 

5,377 

100.0 

Females*US  Navy  Reserve 

12,839 

12,839 

6,693 

100.0 

Females*US  Navy  Reserve*Enlisted 

10,105 

10,105 

4,732 

100.0 

Females*US  Navy  Reserve*  Officers 

2,734 

2,734 

1,961 

100.0 

Females*US  Marine  Corps  Reserve 

1,614 

1,614 

521 

100.0 

Females*US  Marine  Corps  Reserve*Enlisted 

1,309 

1,309 

376 

100.0 

Females*US  Marine  Corps  Reserve*0fficers 

305 

305 

145 

100.0 

Females*US  Air  Force  Reserve 

18,263 

18,263 

9,936 

100.0 

Females*US  Air  Force  Reserve*Enlisted 

14,525 

14,525 

7,604 

100.0 

Females*US  Air  Force  Reserve*Officers 

3,738 

3,738 

2,332 

100.0 

Males 

662,565 

331,283 

60,589 

50.0 

Males*Enlisted 

558,987 

287,319 

45,144 

51.4 

Males*El-E4 

280,993 

166,348 

13,379 

59.2 

Males*E5-E9 

277,994 

120,649 

31,765 

43.4 

Males*Officers 

103,578 

44,228 

15,445 

42.7 

Males*01-03 

46,359 

20,537 

5,744 

44.3 

Males*04-06 

46,387 

19,297 

8,094 

41.6 

Males*TPU 

541,576 

289,202 

44,253 

53.4 

Males*AGR 

61,021 

17,696 

6,738 

29.0 

Males*IMA 

8,903 

5,475 

1,803 

61.5 

Males*Non-FIispanic  White 

464,800 

228,682 

44,383 

49.2 

Males*Total  Minority 

197,765 

102,838 

16,206 

52.0 

Males*National  Guard 

376,829 

171,457 

32,088 

45.5 

Males* Army  National  Guard 

292,616 

130,507 

19,482 

44.6 

Males* Army  National  Guard*Enlisted 

253,791 

117,251 

15,201 

46.2 

Males*Army  National  Guard*Officers 

38,825 

13,317 

4,281 

34.3 

Males* Air  National  Guard 

84,213 

41,096 

12,606 

48.8 

Males* Air  National  Guard*Enlisted 

72,107 

34,611 

10,296 

48.0 

Males*Air  National  Guard*Officers 

12,106 

6,477 

2,310 

53.5 

Males*Reserve 

285,736 

159,726 

28,501 

55.9 

Males*US  Army  Reserve 

152,560 

75,822 

12,713 

49.7 

Males*US  Army  Reserve*Enlisted 

125,700 

65,615 

9,140 

52.2 

Males*US  Army  Reserve*Officers 

26,860 

10,314 

3,573 

38.4 

Males*US  Navy  Reserve 

45,388 

23,420 

5,799 

51.6 

Males*US  Navy  Reserve*Enlisted 

33,601 

18,313 

3,579 

54.5 

Males*US  Navy  Reserve*  Officers 

11,787 

5,104 

2,220 

43.3 

Males*US  Marine  Corps  Reserve 

36,854 

34,753 

3,189 

94.3 
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Domain 

Population 

Approximate 
Sample  Size1 

Expected 

Responses 

Percent 

Sampled 

Males*  US  Marine  Corps  Reserve*Enlisted 

32,894 

30,789 

1,909 

93.6 

Males*US  Marine  Corps  Reserve*Officers 

3,960 

3,960 

1,280 

100.0 

Males*US  Air  Force  Reserve 

50,934 

25,671 

6,800 

50.4 

Males*US  Air  Force  Reserve*Enlisted 

40,894 

20,611 

5,019 

50.4 

Males*US  Air  Force  Reserve*Officers 

10,040 

5,060 

1,781 

50.4 

!This  is  an  approximate  sample  size  that  comes  from  the  sampling  tool  since  not  all  domains  are  explicitly  defined  by  the  stratum  definitions.  For 
example,  since  IMAs  are  not  used  as  a  stratifier,  the  actual  number  of  IMAs  is  not  known  until  the  sample  is  actually  selected 
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Appendix  B. 

Categorical  Variables  Used  for  the  Eligibility 
and  Completion  Adjustments 


Variable  Name 

Categories 

Demographic  Factors 

Family  Status 

Married  With  Child(ren);  Married  Without  Child(ren);  Single  With 
Child(ren);  Single  Without  Child(ren);  Unknown 

Education 

No  College  or  DK;  Some  College;  4-year  Degree;  Grad  /  Professional 
Degree 

Race/Ethnic  Category 

Hispanic;  Non-Hispanic  Black;  Non-minority;  Other  Race;  Unknown 

US  Citizenship  Origin  Code 

Born  outside  the  US,  GU,  PR,  or  VI  to  at  least  one  citizen  parent;  Born 
within  the  US,  GU,  PR  or  VI;  Unknown  or  NA;  US  citizen  by 
naturalization;  US  citizen,  parent  became  a  citizen  by  naturalization 

US  Citizenship  Status  Code 

Born  outside  the  US,  GU,  PR,  or  VI  to  at  least  one  citizen  parent;  Born 
within  the  US,  GU,  PR  or  VI;  Non  US  citizen  or  national;  Unknown  or 

NA 

Military  Career  Factors 

Military  Accession  Program 

Air  National  Guard  Academy  of  Military  Sciences;  Aviation  Cadet 
program;  Aviation  training  program  other  than  OCS,  AOCS,  OTS,  or  PLC; 
Direct  appointment  authority,  Commissioned  Off,  all  other;  Direct 
appointment  authority.  Commissioned  Off,  professional;  Direct 
appointment  authority,  commissioned  warrant  officer;  Induction;  National 
Guard  state  OCS;  OCS,  AOCS,  OTS,  or  PLC;  Other;  ROTC/NROTC 
non-scholarship  program;  ROTC/NROTC  scholarship  program;  S;  US 

Air  Force  Academy;  US  Coast  Guard  Academy;  US  Merchant  Marine 
Academy;  US  Military  Academy;  US  Naval  Academy;  Unknown  or  Not 
Applicable;  Vol  enlist  -  Rsv  Comp  for  Reg  DEP  -  10  USC  12103/10  USC 
513;  Voluntary  enlistment  -  Rsv  Comp,  Sec  511,  ref(b).  Excl  DEP; 
Voluntary  enlistment  in  a  Regular  Component;  Voluntary  enlistment  in  a 
Regular  component  under  the  NCSp;  Warrant  Officer  Aviation  Training 
Program 

Active  Duty  Status 

Not  on  Active  Duty;  On  Active  Duty 

Active  Duty  and  Special  Operations 
Status 

Active;  Active  Special  Operations;  Unknown 

Active  Guard  &  Reserve,  or  Full 
Time  National  Guard  Duty  Statue 

ID 

AGR:  10  USC  10211;  AGR:  10USC  12310;  AGR:  32;  AGR:  Other; 
Military  Technician;  Unknown  or  NA 

Combat  Occupation  Flag 

N;  Y 

Current  Deployment  Status 

Currently  Deployed;  Never  deployed;  Not  currently  deployed 

Deployment  Flag  in  the  Last  12 
months 

Deployed  in  the  last  12  months  or  currently  deployed;  Never  deployed  (as 
of  March  2015);  Not  deployed  in  the  last  12  months 

Deployment  Flag  in  the  Last  24 
months 

Deployed  in  the  last  24  months  or  currently  deployed;  Never  deployed  (as 
of  March  2015);  Not  deployed  in  the  last  24  months 

Eligibility  Status  as  of  August  2015 

Eligible,  Ineligible 

Paygrade 

E01;  E02;  E03;  E04;  E05;  E06;  E07;  E08;  E09;  W01;  W02; 

W03;  W04;  W05;  O01;  002;  003  ;  004;  005  ;  006 

Primary  Regular  Component 

Service  Indicator 

Yes;  No;  Unknown  or  NA; 

Reserve  Category  Programs 

AGR/TAR;  IMA;  Military  Technicians;  TPU;  Unknown 

Reserve  Organization  Code 

Air  Force  Reserve;  Air  National  Guard;  Army  National  Guard;  Army 
Reserve;  Marine  Corps  Reserve;  Navy  Reserve 

Reserve  Category  Group  Code 

Active  Guard/Reserve  (AGR);  Military  Technician  (MILTECH);  Selected 
Reserve  (not  including  AGR  or  MILTECH) 

Reserve  Subcategory  Code 

Active  Guard  Reserve;  Awaiting  Second  Part  of  IADT;  Drilling  Unit 
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Variable  Name  Categories 

Member;  FT  members  performing  AD  on  FTNGD  for  >180,  but  exempt 
from;  Individual  Mobilization  Augmentees  (IMA);  On  Initial  Active  Duty 
For  Training  (IADT);  Person  awaiting  IADT;  SEL  RES-Other  Training 
Program;  Simultaneous  Membership  Program  (SMP) 


Reserve  Category  Code 

Selected  Reserve  -  Trained  in  Units;  Selected  Reserve  -  Trained  individuals 
(non-unit);  Selected  Reserve  -  Training  Pipeline 

Reserve  Component  Category  Code 

Unknown  or  NA;  Inactive  National  Guard,  RAPIDS  entry;  Inactive  National 
Guard,  individual;  Ready  Reserve  training,  individual  in  officer  training 
program;  Ready  Reserve  training,  individual  in  Flealth  Professional 
Scholarship  program;  Reserve  Officer  Training  Corps  (ROTC);  Individual 
Ready  Reserve,  RAPIDS  entry;  Individual  Ready  Reserve,  trained; 

Individual  Ready  Reserve,  awaiting  IADT,  not  authorized  to  perform  IDT; 
Selected  Reserve,  Unknown  Cat.;  Selected  Reserve,  trained  individual  in 
unit,  48  or  more  IDT  periods;  Selected  Reserve,  trained  individual  in  unit. 
Active  Guard  or  Reserve;  Full-Time  Members  (Special  Category):  Trained 
Selected  Reserve  members  who  are  performing  AD  for  more  than  180  days 
in  a  fiscal  year,  but  who  are  exempted  from  counting  against  the  AD 
strengths;  Selected  Reserve,  trained  individual  not  in  unit.  Individual 
Mobilization  Augmentee;  Selected  Reserve,  individual  in  training  pipeline, 
on  IADT ;  Selected  Reserve,  individual  in  training  pipeline,  awaiting  IADT, 
authorized  to  perform  IDT ;  Selected  Reserve,  individual  in  training  pipeline, 
awaiting  second  part  of  IADT;  Selected  Reserve,  individual  in  training 
pipeline,  Simultaneous  Membership  Program;  Selected  Reserve,  individual 
in  training  pipeline,  other  training  program;  Standby  Reserve  (Y9);  Standby 
Reserve,  individual  on  Active  Status  list;  Standby  Reserve,  individual  on 
Inactive  Status  List,  20  or  more  years  Reserve  service  and  less  than  30% 
disability;  Standby  Reserve,  individual  on  Inactive  Status  List,  other; 

Reserve  Category  Unknown 

Military  Environment  Factors 

Assigned  UIC  Change  of  Station 
Flag  as  of  March  2014 

N;  Y 

Assigned  UIC  Change  of  Station 
Flag  as  of  August  2015 

N;  Y 

Duty  UIC  Change  of  Station  Flag  as 
of  March  2014 

N;  Y 

Duty  UIC  Change  of  Station  Flag  as 
of  August  2015 

N;  Y 

Survey  Fielding  Factors 

First  Letter  Returned  as  PND 

N;  Y 

Invalid  Army  E-mail  Address  Flag 

@Army.mil  address;  Not  @Army.mil  Address 

E-mail  Address  Purchase  Flag 

Do  Not  Purchase  e-mail;  Purchase  E-mail 

E-mail  Address  Flag 

At  least  one  e-mail  address;  No  e-mail  address 

Flome  Address  Flag 

Address  available;  No  Flome  Address 

Change  in  Mailing  Address  since 
Sample  Frame  Development 

N;  Y 

Mail  Address  Flag 

N;  Y 

Number  of  E-mail  Addresses 

No  e-mail  address;  One  e-mail  address;  Two  e-mail  addresses 
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Appendix  C. 

Distribution  of  Weights  and  Adjustment 
Factors  by  Eligibility  Status  for  Female 


Eligibility 

Status 

Statistic 

Sampling 

Weight 

Eligibility 

Status 

Adjusted 

Weight 

Complete 

Eligible 

Response 

Adjusted 

Weight 

Final  Weight 
With 

Nonresponse 
and  Raking 
Factors 

Eligibility 

Status 

Factor 

Complete 

Eligible 

Response 

Factor 

Raking 

Factor 

Eligible 

Respondents 

N 

34,706 

34,706 

34,706 

34,706 

34,706 

34,706 

34,706 

MIN 

1.00 

1.14 

1.17 

1.18 

1.14 

1.01 

0.77 

MAX 

1.00 

38.18 

39.19 

41.23 

38.18 

1.07 

1.21 

MEAN 

1.00 

4.04 

4.16 

4.34 

4.04 

1.03 

1.03 

STD 

0.00 

3.24 

3.34 

3.64 

3.24 

0.01 

0.06 

CV 

0.00 

0.80 

0.80 

0.84 

0.80 

0.01 

0.06 

Eligible, 

Incomplete 

Response 

N 

987 

987 

987 

987 

987 

987 

987 

MIN 

1.00 

1.18 

0.00 

0.00 

1.18 

0.00 

MAX 

1.00 

27.27 

0.00 

0.00 

27.27 

0.00 

MEAN 

1.00 

4.23 

0.00 

0.00 

4.23 

0.00 

STD 

0.00 

3.42 

0.00 

0.00 

3.42 

0.00 

CV 

0.00 

0.81 

0.81 

Self/Proxy 

Ineligibles 

N 

523 

523 

523 

523 

523 

523 

523 

MIN 

1.00 

1.43 

1.43 

1.48 

1.43 

1.00 

0.77 

MAX 

1.00 

43.06 

43.06 

44.83 

43.06 

1.00 

1.18 

MEAN 

1.00 

6.60 

6.60 

6.99 

6.60 

1.00 

1.04 

STD 

0.00 

5.51 

5.51 

5.99 

5.51 

0.00 

0.06 

CV 

0.00 

0.84 

0.84 

0.86 

0.84 

0.00 

0.05 

Nonresponde 

nts 

N 

115,035 

115,035 

115,035 

115,035 

115,035 

115,035 

115,035 

MIN 

1.00 

0.00 

0.00 

0.00 

0.00 

MAX 

1.00 

0.00 

0.00 

0.00 

0.00 

MEAN 

1.00 

0.00 

0.00 

0.00 

0.00 

STD 

0.00 

0.00 

0.00 

0.00 

0.00 

CV 

0.00 

Record 

Ineligibles 

N 

3,191 

3,191 

3,191 

3,191 

3,191 

3,191 

3,191 

MIN 

1.00 

1.00 

1.00 

0.00 

MAX 

1.00 

1.00 

1.00 

0.00 

MEAN 

1.00 

1.00 

1.00 

0.00 

STD 

0.00 

0.00 

0.00 

0.00 

CV 

0.00 

0.00 

0.00 
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Appendix  D. 

Distribution  of  Weights  and  Adjustment 
Factors  by  Eligibility  Status  for  Male 


Eligibility 

Status 

Statistic 

Sampling 

Weight 

Eligibility 

Status 

Adjusted 

Weight 

Complete 

Eligible 

Response 

Adjusted 

Weight 

Final  Weight 
With 

Nonresponse 
and  Raking 
Factors 

Eligibility 

Status 

Factor 

Complete 

Eligible 

Response 

Factor 

Raking 

Factor 

Eligible 

Respondents 

N 

52,421 

52,421 

52,421 

52,421 

52,421 

52,421 

52,421 

MIN 

1.00 

1.19 

1.21 

1.58 

1.07 

1.02 

0.87 

MAX 

5.69 

121.51 

125.96 

135.92 

64.72 

1.12 

1.35 

MEAN 

2.32 

11.53 

11.77 

12.24 

5.72 

1.02 

1.03 

STD 

0.91 

9.03 

9.26 

9.98 

5.26 

0.00 

0.09 

CV 

0.39 

0.78 

0.79 

0.81 

0.92 

0.00 

0.09 

Eligible, 

Incomplete 

Response 

N 

998 

998 

998 

998 

998 

998 

998 

MIN 

1.00 

1.79 

0.00 

0.00 

1.32 

0.00 

MAX 

5.69 

57.32 

0.00 

0.00 

30.53 

0.00 

MEAN 

2.29 

12.69 

0.00 

0.00 

6.38 

0.00 

STD 

0.93 

9.57 

0.00 

0.00 

5.59 

0.00 

CV 

0.41 

0.75 

0.88 

Self/Proxy 

Ineligibles 

N 

1,018 

1,018 

1,018 

1,018 

1,018 

1,018 

1,018 

MIN 

1.00 

2.53 

2.53 

3.08 

1.40 

1.00 

0.88 

MAX 

5.69 

114.73 

114.73 

123.58 

64.72 

1.00 

1.34 

MEAN 

2.12 

19.41 

19.41 

20.45 

10.33 

1.00 

1.05 

STD 

0.75 

15.88 

15.88 

16.95 

9.22 

0.00 

0.09 

CV 

0.35 

0.82 

0.82 

0.83 

0.89 

0.00 

0.09 

Nonrespondents 

N 

269,456 

269,456 

269,456 

269,456 

269,456 

269,456 

269,456 

MIN 

1.00 

0.00 

0.00 

0.00 

0.00 

MAX 

5.69 

0.00 

0.00 

0.00 

0.00 

MEAN 

1.94 

0.00 

0.00 

0.00 

0.00 

STD 

0.59 

0.00 

0.00 

0.00 

0.00 

CV 

0.30 

Record 

Ineligibles 

N 

7,439 

7,439 

7,439 

7,439 

7,439 

7,439 

7,439 

MIN 

1.00 

1.00 

1.00 

0.00 

MAX 

5.69 

5.69 

5.69 

0.00 

MEAN 

1.84 

1.84 

1.84 

0.00 

STD 

0.52 

0.52 

0.52 

0.00 

CV 

0.28 

0.28 

0.28 
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